Initial commit for our basic Postgres Operator UI:

* Create and modify Postgres manifests
 * Watch Operator Logs in the UI
 * Observe cluster creation progress
 * S3 Backup browser for clone and restore

Many thanks to Manuel Gomez and Jan Mussler for the initial UI work a long time ago!
This commit is contained in:
Felix Kunde 2019-07-12 14:34:38 +02:00
parent 4fc5822b24
commit 32d202e346
60 changed files with 12752 additions and 13 deletions

View File

@ -28,6 +28,7 @@ There is a browser-friendly version of this documentation at
* [How it works](docs/index.md)
* [The Postgres experience on K8s](docs/user.md)
* [The Postgres Operator UI](docs/operator-ui.md)
* [DBA options - from RBAC to backup](docs/administrator.md)
* [Debug and extend the operator](docs/developer.md)
* [Configuration options](docs/reference/operator_parameters.md)

View File

@ -57,3 +57,38 @@ pipeline:
fi
export IMAGE
make push
- id: "build-operator-ui"
type: "script"
commands:
- desc: "Prepare environment"
cmd: |
apt-get update
apt-get install -y build-essential
- desc: "Compile JavaScript app"
cmd: |
cd ui
make appjs
- desc: "Build and push Docker image"
cmd: |
cd ui
image_base='registry-write.opensource.zalan.do/acid/postgres-operator-ui'
if [[ "${CDP_TARGET_BRANCH}" == 'master' && -z "${CDP_PULL_REQUEST_NUMBER}" ]]
then
image="${image_base}"
else
image="${image_base}-test"
fi
image_with_tag="${image}:c${CDP_BUILD_VERSION}"
if docker pull "${image}"
then
docker build --cache-from="${image}" -t "${image_with_tag}" .
else
docker build -t "${image_with_tag}" .
fi
docker push "${image_with_tag}"

View File

@ -313,7 +313,7 @@ empty sequence `[]`. Setting the field to `null` or omitting it entirely may
lead to K8s removing this field from the manifest due to its
[handling of null fields](https://kubernetes.io/docs/concepts/overview/object-management-kubectl/declarative-config/#how-apply-calculates-differences-and-merges-changes).
Then the resultant manifest will not contain the necessary change, and the
operator will respectively do noting with the existing source ranges.
operator will respectively do nothing with the existing source ranges.
## Running periodic 'autorepair' scans of K8s objects
@ -409,3 +409,40 @@ A secret can be pre-provisioned in different ways:
* Generic secret created via `kubectl create secret generic some-cloud-creds --from-file=some-cloud-credentials-file.json`
* Automatically provisioned via a custom K8s controller like
[kube-aws-iam-controller](https://github.com/mikkeloscar/kube-aws-iam-controller)
## Setting up the Postgres Operator UI
With the v1.2 release the Postgres Operator is shipped with a browser-based
configuration user interface (UI) that simplifies managing Postgres clusters
with the operator. The UI runs with Node.js and comes with it's own docker
image.
Run NPM to continuously compile `tags/js` code. Basically, it creates an
`app.js` file in: `static/build/app.js`
```
(cd ui/app && npm start)
```
To build the Docker image open a shell and change to the `ui` folder. Then run:
```
docker build -t registry.opensource.zalan.do/acid/postgres-operator-ui:v1.2.0 .
```
Apply all manifests for the `ui/manifests` folder to deploy the Postgres
Operator UI on K8s. For local tests you don't need the Ingress resource.
```
kubectl apply -f ui/manifests
```
Make sure the pods for the operator and the UI are both running. For local
testing you need to apply proxying and port forwarding so that the UI can talk
to the K8s and Postgres Operator REST API. You can use the provided
`run_local.sh` script for this. Make sure it uses the correct URL to your K8s
API server, e.g. for minikube it would be `https://192.168.99.100:8443`.
```
./run_local.sh
```

Binary file not shown.

After

Width:  |  Height:  |  Size: 44 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 64 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 36 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 127 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 131 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 254 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 116 KiB

65
docs/operator-ui.md Normal file
View File

@ -0,0 +1,65 @@
# Postgres Operator UI
The Postgres Operator UI provides a graphical interface for a convenient
database-as-a-service user experience. Once the operator is set up by database
and/or Kubernetes (K8s) admins it's very easy for other teams to create, clone,
watch, edit and delete their own Postgres clusters. Information on the setup
and technical details can be found in the [admin docs](administrator.md#setting-up-the-postgres-operator-ui).
## Create a new cluster
In the top menu select the "New cluster" option and adjust the values in the
text fields. The cluster name is composed of the team plus given name. Among the
available options are [enabling load balancers](administrator.md#load-balancers-and-allowed-ip-ranges),
[volume size](user.md#increase-volume-size),
[users and databases](user.md#manifest-roles) and
[pod resources](cluster-manifest.md#postgres-container-resources).
![pgui-new-cluster](diagrams/pgui-new-cluster.png "Create a new cluster")
On the left side you will see a preview of the Postgres cluster manifest which
is applied when clicking on the green "Create cluster" button.
## Cluster starting up
After the manifest is applied to K8s the Postgres Operator will create all
necessary resources. The progress of this process can nicely be followed in UI
status page.
![pgui-cluster-startup](diagrams/pgui-cluster-startup.png "Cluster starting up")
![pgui-waiting-for-master](diagrams/pgui-waiting-for-master.png "Waiting for master pod")
Usually, the startup should only take up to 1 minute. If you feel the process
got stuck click on the "Logs" button to inspect the operator logs. From the
"Status" field in the top menu you can also retrieve the logs and queue of each
worker the operator is using. The number of concurrent workers can be
[configured](reference/operator_parameters.md#general).
![pgui-operator-logs](diagrams/pgui-operator-logs.png "Checking operator logs")
Once the startup has finished you will see the cluster address path. When load
balancers are enabled the listed path can be used as the host name when
connecting to PostgreSQL. But, make sure your IP is within the specified
`allowedSourceRanges`.
![pgui-finished-setup](diagrams/pgui-finished-setup.png "Status page of ready cluster")
## Update and delete clusters
Created clusters are listed under the menu "PostgreSQL clusters". You can get
back to cluster's status page via the "Status" button. From both menus you can
choose to edit the manifest, [clone](user.md#how-to-clone-an-existing-postgresql-cluster)
or delete a cluster.
![pgui-cluster-list](diagrams/pgui-cluster-list.png "List of PostgreSQL clusters")
Note, that not all [manifest options](reference/cluster_manifest.md) are yet
supported in the UI. If you try to add them in the editor view it won't have an
effect. Use `kubectl` commands instead. The displayed manifest on the left side
will also show parameters patched that way.
When deleting a cluster you are asked to type in its namespace and name to
confirm the action.
![pgui-delete-cluster](diagrams/pgui-delete-cluster.png "Confirm cluster deletion")

View File

@ -1,4 +1,4 @@
## Command-line options
# Command-line options
The following command-line options are supported for the operator:

View File

@ -2,15 +2,14 @@ site_name: Postgres Operator
repo_url: https://github.com/zalando/postgres-operator
theme: readthedocs
pages:
- Introduction: index.md
- Quickstart: quickstart.md
- Administrator Guide: administrator.md
- User Guide: user.md
- Developer Guide: developer.md
nav:
- index.md
- quickstart.md
- operator-ui.md
- administrator.md
- user.md
- developer.md
- Reference:
- Operator Configuration: reference/operator_parameters.md
- Cluster Manifest description: reference/cluster_manifest.md
- Command-line options and environment: reference/command_line_and_environment.md
- reference/operator_parameters.md
- reference/cluster_manifest.md
- reference/command_line_and_environment.md

10
ui/.dockerignore Normal file
View File

@ -0,0 +1,10 @@
*#
*.pyc
*~
.*.sw?
.git
__pycache__
app/node_modules
operator_ui/static/build/*.hot-update.js
operator_ui/static/build/*.hot-update.json

40
ui/Dockerfile Normal file
View File

@ -0,0 +1,40 @@
FROM alpine:3.6
MAINTAINER team-acid@zalando.de
EXPOSE 8080
RUN \
apk add --no-cache \
alpine-sdk \
autoconf \
automake \
ca-certificates \
libffi-dev \
libtool \
python3 \
python3-dev \
zlib-dev \
&& \
python3 -m ensurepip && \
rm -r /usr/lib/python*/ensurepip && \
pip3 install --upgrade \
gevent \
jq \
pip \
setuptools \
&& \
rm -rf \
/root/.cache \
/tmp/* \
/var/cache/apk/*
COPY requirements.txt /
RUN pip3 install -r /requirements.txt
COPY operator_ui /operator_ui
ARG VERSION=dev
RUN sed -i "s/__version__ = .*/__version__ = '${VERSION}'/" /operator_ui/__init__.py
WORKDIR /
ENTRYPOINT ["/usr/bin/python3", "-m", "operator_ui"]

3
ui/MANIFEST.in Normal file
View File

@ -0,0 +1,3 @@
recursive-include operator_ui/static *
recursive-include operator_ui/templates *
include *.rst

43
ui/Makefile Normal file
View File

@ -0,0 +1,43 @@
.PHONY: clean test appjs docker push mock
BINARY ?= postgres-operator-ui
BUILD_FLAGS ?= -v
CGO_ENABLED ?= 0
ifeq ($(RACE),1)
BUILD_FLAGS += -race -a
CGO_ENABLED=1
endif
LOCAL_BUILD_FLAGS ?= $(BUILD_FLAGS)
LDFLAGS ?= -X=main.version=$(VERSION)
DOCKERDIR = docker
IMAGE ?= registry.opensource.zalan.do/acid/$(BINARY)
VERSION ?= $(shell git describe --tags --always --dirty)
TAG ?= $(VERSION)
GITHEAD = $(shell git rev-parse --short HEAD)
GITURL = $(shell git config --get remote.origin.url)
GITSTATU = $(shell git status --porcelain || echo 'no changes')
TTYFLAGS = $(shell test -t 0 && echo '-it')
default: docker
clean:
rm -fr operator_ui/static/build
test:
tox
appjs:
docker run $(TTYFLAGS) -u $$(id -u) -v $$(pwd):/workdir -w /workdir/app node:10.1.0-alpine npm install
docker run $(TTYFLAGS) -u $$(id -u) -v $$(pwd):/workdir -w /workdir/app node:10.1.0-alpine npm run build
docker: appjs
docker build --build-arg "VERSION=$(VERSION)" -t "$(IMAGE):$(TAG)" .
@echo 'Docker image $(IMAGE):$(TAG) can now be used.'
push: docker
docker push "$(IMAGE):$(TAG)"
mock:
docker run -it -p 8080:8080 "$(IMAGE):$(TAG)" --mock

1
ui/app/.eslintignore Normal file
View File

@ -0,0 +1 @@
src/vendor/*.js

27
ui/app/.eslintrc.yml Normal file
View File

@ -0,0 +1,27 @@
parserOptions:
sourceType: module
env:
browser: true
node: true
es6: true
extends: 'eslint:recommended'
rules:
indent:
- error
- 4
linebreak-style:
- error
- unix
quotes:
- error
- single
prefer-const:
- error
no-redeclare:
- error
no-unused-vars:
- warn
- argsIgnorePattern: "^_"
semi:
- error
- never

14
ui/app/README.rst Normal file
View File

@ -0,0 +1,14 @@
This directory contains the EcmaScript frontend code of the PostgreSQL Operator UI and is only needed during build time.
The JavaScript application bundle (webpack) will be generated to ``operator_ui/static/build/app*.js`` by running:
.. code-block:: bash
$ npm install
$ npm run build
Frontend development is supported by watching the source code and continuously recompiling the webpack:
.. code-block:: bash
$ npm start

6924
ui/app/package-lock.json generated Normal file

File diff suppressed because it is too large Load Diff

54
ui/app/package.json Normal file
View File

@ -0,0 +1,54 @@
{
"name": "postgres-operator-ui",
"version": "1.0.0",
"description": "PostgreSQL Operator UI",
"main": "src/app.js",
"config": {
"buildDir": "../operator_ui/static/build"
},
"scripts": {
"prestart": "npm install",
"start": "NODE_ENV=development webpack --watch",
"webpack": "webpack --config ./webpack.config.js",
"build": "NODE_ENV=development npm run webpack",
"prewebpack": "npm run clean",
"lint": "eslint ./src/**/*.js",
"clean": "rimraf $npm_package_config_buildDir && mkdir $npm_package_config_buildDir"
},
"repository": {
"type": "git",
"url": "git+https://github.com/zalando/postgres-operator.git"
},
"author": "",
"license": "ISC",
"bugs": {
"url": "https://github.com/zalando/postgres-operator.git/issues"
},
"homepage": "https://github.com/zalando/postgres-operator.git#readme",
"dependencies": {
"@babel/core": "^7.0.0-beta.46",
"@babel/polyfill": "^7.0.0-beta.46",
"@babel/runtime": "^7.0.0-beta.46",
"pixi.js": "^4.7.3"
},
"devDependencies": {
"@babel/plugin-transform-runtime": "^7.0.0-beta.46",
"@babel/preset-env": "^7.0.0-beta.46",
"babel-loader": "^8.0.0-beta.2",
"brfs": "^1.6.1",
"dedent-js": "1.0.1",
"eslint": "^4.19.1",
"eslint-loader": "^1.6.1",
"js-yaml": "3.13.1",
"pug": "^2.0.3",
"rimraf": "^2.5.4",
"riot": "^3.9.5",
"riot-hot-reload": "1.0.0",
"riot-route": "^3.1.3",
"riot-tag-loader": "2.0.2",
"sort-by": "^1.2.0",
"transform-loader": "^0.2.3",
"webpack": "^4.28.2",
"webpack-cli": "^3.1.2"
}
}

240
ui/app/src/app.js Normal file
View File

@ -0,0 +1,240 @@
import jQuery from 'jquery'
import riot from 'riot'
import 'riot-hot-reload'
import './edit.tag.pug'
import './postgresql.tag.pug'
import './help-edit.tag.pug'
import './help-general.tag.pug'
import './postgresqls.tag.pug'
import './logs.tag.pug'
import './new.tag.pug'
import './status.tag.pug'
import './app.tag.pug'
import './restore.tag.pug'
Object.fromEntries = entries => entries.length === 0 ? {} : Object.assign(...entries.map(([k, v]) => ({[k]: v})))
Object.mapValues = (o, f) => Object.fromEntries(Object.entries(o).map(([k, v]) => [k, f(v, k)]))
Object.mapEntries = (o, f) => Object.fromEntries(Object.entries(o).map(f).filter(x => x))
Object.filterEntries = (o, f) => Object.mapEntries(o, entry => f(entry) && entry)
Object.filterValues = (o, f) => Object.filterEntries(o, ([key, value]) => f(value) && [key, value])
const getDefaulting = (object, key, def) => (
object.hasOwnProperty(key) ? object[key] : def
)
const Dynamic = (options={}) => {
const instance = {
init: getDefaulting(options, 'init', () => ''),
refresh: getDefaulting(options, 'refresh', () => true),
update: getDefaulting(options, 'update', value => (instance.state = value, true)),
validState: getDefaulting(options, 'validState', state => (
state !== undefined &&
state !== null &&
typeof state === 'string' &&
state.length > 0
)),
edit: event => (instance.update(event.target.value, instance, event), true),
valid: () => instance.validState(instance.state),
}
instance.state = instance.init()
return instance
}
/*
Dynamics manages a dynamic array whose elements are themselves Dynamic objects.
The default initializer builds an empty array as the initial state.
The "add" DOM event callback is provided to add a newly initialized Dynamic
object to the end of the state array. The Dynamic array item is initialized
with the "itemInit" callback, which can be specified with a constructor option
and defaults to creating a Dynamic with all default options.
The "remove" DOM event callback is provided to handle DOM events that should
or remove a specific item. For events on elements generated by iterating the
state with an each= attribute, the event.item will be set to the correct value.
The refresh callback is forwarded to all constituent Dynamic objects.
*/
const Dynamics = (options={}) => {
const instance = Object.assign(
Dynamic(
Object.assign(
{ init: () => [] },
'refresh' in options
? { refresh: options.refresh }
: undefined
)
),
{
itemInit: getDefaulting(options, 'itemInit', () =>
Dynamic(
'refresh' in options
? { refresh: options.refresh }
: {}
)
),
itemValid: getDefaulting(options, 'itemValid', item => item.valid()),
validState: state => state.every(instance.itemValid),
update: () => true,
edit: () => true,
add: _event => {
instance.state.push(instance.itemInit())
instance.refresh()
return true
},
remove: event => {
instance.state.splice(instance.state.indexOf(event.item), 1)
instance.refresh()
return true
},
}
)
Object.defineProperty(instance, 'valids', { get: () =>
instance.state.filter(instance.itemValid)
})
return instance
}
/*
DynamicSet manages a keyed collection of Dynamic objects. The constructor
receives an object mapping keys to initialization functions, and its state is a
mapping from the same keys to Dynamics initialized using the corresponding
key's initialization function. A DynamicSet is valid when its constituent
Dynamics are all simultaneously valid. The refresh callback is forwarded to
all constituent Dynamic objects.
Example:
DynamicSet({
foo: undefined,
bar: () => 'baz',
})
This call would create a DynamicSet with two constituent dynamics in its state:
one of them under the 'foo' key of the state object, built with the default
Dynamic initializer, and another under the 'bar' key of the state object, whose
state, in turn, would initially hold the value 'baz'.
*/
const DynamicSet = (items, options={}) => Object.assign(
Dynamic(
Object.assign(
{
init: () => Object.mapValues(items, init =>
Dynamic(
Object.assign(
init ? { init: init } : undefined,
'refresh' in options ? { refresh: options.refresh } : undefined
)
)
),
},
'refresh' in options
? { refresh: options.refresh }
: undefined
)
),
{
items: items,
validState: state => Object.values(state).every(item => item.valid()),
edit: () => true,
update: () => true,
}
)
const delete_cluster = (namespace, clustername) => {
jQuery.confirm({
backgroundDismiss: true,
content: `
<p>
Are you sure you want to remove this PostgreSQL cluster? If so,
please <strong>type the cluster name here
(<code>${namespace}/${clustername}</code>)</strong> and click the
confirm button:
</p>
<input
type="text"
class="confirm-delete"
placeholder="cluster name"
style="width: 100%"
>
<hr>
<p><small>
<strong>Note</strong>: if you create a cluster with the same name as
this one after deleting it, the new cluster will restore the data
from this cluster's current backups stored in AWS S3. This behavior
will change soon and you will be able to reuse a cluster name and
get a completely new cluster.
</small></p>
`,
escapeKey: true,
icon: 'glyphicon glyphicon-warning-sign',
title: 'Confirm cluster deletion?',
typeAnimated: true,
type: 'red',
onOpen: function () {
const dialog = this
const confirm = dialog.buttons.confirm
const confirmSelector = jQuery(confirm.el)
const input = dialog.$content.find('input')
input.on('input', () => {
if (input.val() === namespace + '/' + clustername) {
confirmSelector.removeClass('btn-default').addClass('btn-danger')
confirm.enable()
} else {
confirm.disable()
confirmSelector.removeClass('btn-danger').addClass('btn-default')
}
})
},
buttons: {
cancel: {
text: 'Cancel',
},
confirm: {
btnClass: 'btn-default',
isDisabled: true,
text: 'Delete cluster',
action: () => {
jQuery.ajax({
type: 'DELETE',
url: (
'/postgresqls/'
+ encodeURI(namespace)
+ '/' + encodeURI(clustername)
),
dataType: 'text',
success: () => location.assign('/#/list'),
error: (r, status, error) => location.assign('/#/list'), // TODO: show error
})
},
},
}
})
}
/* Unfortunately, there does not appear to be a good way to import local modules
inside a Riot tag, so we define/import things here and pass them manually in the
opts variable. Remember to propagate opts manually when instantiating tags. */
riot.mount('app', {
Dynamic: Dynamic,
Dynamics: Dynamics,
DynamicSet: DynamicSet,
delete_cluster: delete_cluster,
})

208
ui/app/src/app.tag.pug Normal file
View File

@ -0,0 +1,208 @@
app
nav.navbar.navbar-inverse.navbar-fixed-top
.container
.navbar-header
a.navbar-brand(href='/')
| PostgreSQL Operator UI
#navbar.navbar-collapse.collapse
ul.nav.navbar-nav
li(class='{ active: ["EDIT", "LIST", "LOGS", "STATUS"].includes(activenav) }')
a(href='/#/list') PostgreSQL clusters
li(class='{ active: "BACKUPS" === activenav }')
a(href='/#/backups') Backups
li(class='{ active: "OPERATOR" === activenav }')
a(href='/#/operator') Status
li(class='{ active: "NEW" === activenav }')
a(href='/#/new') New cluster
li(if='{ config }')
a(href='{ config.docs_link }' target='_blank') Documentation
.container-fluid
.alert.alert-warning.alert-dismissible(
if='{ config && config.kubernetes_in_maintenance }'
role='alert'
)
button.close(
aria-label='Close'
data-dismiss='alert'
type='button'
)
span.glyphicon.glyphicon-remove(aria-hidden='true')
p.lead
span.glyphicon.glyphicon-exclamation-sign(aria-hidden='true')
span.sr-only Warning:
|
| This Kubernetes cluster appears to be undergoing maintenance. You may experience delays in database cluster creation and changes.
.sk-spinner-pulse(
if='{ config !== null && teams !== null && (config === undefined || teams === undefined) }'
)
p(if='{ config === null || teams === null }')
| Error loading UI configuration. Please
|
a(onclick="window.location.reload(true);") try again
|
| or
|
a(href="/") start over
| .
div(if='{ config }')
edit(
if='{ activenav === "EDIT" }'
clustername='{ clustername }'
config='{ config }'
namespace='{ namespace }'
opts='{ opts }'
read_write='{ read_write }'
teams='{ teams }'
)
logs(
if='{ activenav === "LOGS"}'
clustername='{ clustername }'
config='{ config }'
namespace='{ namespace }'
opts='{ opts }'
read_write='{ read_write }'
teams='{ teams }'
)
new(
if='{ activenav === "NEW" }'
backup_name='{ backup_name }'
backup_timestamp='{ backup_timestamp }'
backup_uid='{ backup_uid }'
config='{ config }'
opts='{ opts }'
read_write='{ read_write }'
teams='{ teams }'
)
postgresql(
if='{ activenav === "STATUS" }'
clustername='{ clustername }'
config='{ config }'
namespace='{ namespace }'
opts='{ opts }'
read_write='{ read_write }'
teams='{ teams }'
)
postgresqls(
if='{ activenav === "LIST" }'
config='{ config }'
opts='{ opts }'
read_write='{ read_write }'
teams='{ teams }'
)
restore(
if='{ activenav === "BACKUPS" }'
config='{ config }'
opts='{ opts }'
read_write='{ read_write }'
teams='{ teams }'
)
status(
if='{ activenav === "OPERATOR" }'
config='{ config }'
opts='{ opts }'
read_write='{ read_write }'
teams='{ teams }'
)
script.
this.config = undefined
this.teams = undefined
this.activenav = 'INIT'
this.read_write = false
const nav = (path, page, parameters, f) => {
route(path, (...args) => {
parameters && parameters.forEach((parameter, index) =>
this[parameter] = args[index]
)
this.activenav = page
this.update()
f && f(...args)
})
}
const navs = (paths, ...args) => (
paths.forEach(path =>
nav(path, ...args)
)
)
;(
jQuery
.get('/config')
.done(config => {
this.config = config
;(
jQuery
.get('/teams')
.done(teams => {
this.teams = teams.sort()
this.team = this.teams[0]
this.read_write = (
this.config
&& (
!this.config.read_only_mode
|| this.teams.includes(this.config.superuser_team)
)
)
nav('/backups', 'BACKUPS')
nav('/edit/*/*', 'EDIT', ['namespace', 'clustername'])
nav('/list', 'LIST')
nav('/logs/*/*', 'LOGS', ['namespace', 'clustername'])
nav('/operator..', 'OPERATOR')
nav('/status/*/*', 'STATUS', ['namespace', 'clustername'])
nav(
'/new',
'NEW',
['backup_name', 'backup_uid', 'backup_timestamp'],
() => this.tags['new'].reset_form(),
)
navs(
[
'/clone/*/*',
'/clone/*/*/*',
],
'NEW',
['backup_name', 'backup_uid', 'backup_timestamp']
)
route.start(true)
if (this.activenav === 'INIT') {
route(this.read_write ? '/new' : '/list')
}
})
.fail(() => this.teams = null)
.always(() => this.update())
)
})
.fail(() => this.config = null)
.always(() => this.update())
)

205
ui/app/src/edit.tag.pug Normal file
View File

@ -0,0 +1,205 @@
edit
.container-fluid
h1.page-header
nav(aria-label="breadcrumb")
ol.breadcrumb
li.breadcrumb-item
a(href='/#/list')
| PostgreSQL clusters
li.breadcrumb-item(if='{ cluster_path }')
a(href='/#/status/{ cluster_path }')
| { qname }
li.breadcrumb-item.active(
aria-current='page'
if='{ cluster_path }'
)
a(href='/#/edit/{ cluster_path }')
| Edit
.row(if='{ cluster_path }')
.col-lg-4
h2 Cluster YAML definition
div
pre
code.language-yaml(ref='yamlNice')
div
.col-lg-5
h3 Edit supported properties
textarea.textarea(
style='width: 100%; height: 200px; font-family: monospace'
ref='changedProperties'
onkeyup='{ updateEditable }'
onchange='{ updateEditable }'
)
| { editablePropertiesText }
h3 Preview changes to spec
pre
code.language-yaml(ref='yamlEditable')
| { editablePropertiesPreview }
button.btn.btn-success(
if='{ opts.read_write }'
value='Save changes'
onclick='{ saveChanges }'
)
| Apply changes
.alert.alert-danger(if='{ saveResult === false }')
b Error
|
| { saveMessage }
.alert.alert-success(if='{ saveResult === true }')
b Changes applied
|
| updates to cluster pending
.col-lg-3
help-edit(config='{ opts.config }')
script.
const yamlParser = require('js-yaml')
this.updateEditable = e => {
if (this.refs.changedProperties.value) {
this.editablePropertiesPreview = yamlParser.safeDump(
yamlParser.safeLoad(
this.refs.changedProperties.value,
),
)
}
}
this.saveChanges = e => {
this.saving = true
this.saveResult = null
this.saveMessage = ''
jsonPayload = JSON.stringify(
yamlParser.safeLoad(
this.refs.changedProperties.value,
),
)
jQuery.ajax({
type: 'POST',
url: '/postgresqls/' + this.cluster_path,
contentType:"application/json",
data: jsonPayload,
processData: false,
success: ()=> {
this.saveResult = true
this.update()
this.pollProgress()
},
error: (r, status, error) => {
this.saveResult = false
this.saveMessage = r.responseJSON.error
this.update()
this.pollProgress()
},
dataType: "json",
})
}
this.pollProgress = () => {
jQuery.get(
'/postgresqls/' + this.cluster_path,
).then(data => {
// Input data:
const i = {}
this.progress.thirdParty = true
i.postgresql = this.progress.thirdPartySpec = data
i.metadata = i.postgresql.metadata
i.spec = i.postgresql.spec
if (i.metadata.selfLink) { delete i.metadata.selfLink }
if (i.metadata.uid) { delete i.metadata.uid }
if (i.metadata.resourceVersion) { delete i.metadata.resourceVersion }
this.update()
this.refs.yamlNice.innerHTML = yamlParser.safeDump(i.postgresql, {sortKeys: true})
// Output data:
const o = this.editableProperties = { spec: {} }
o.spec.allowedSourceRanges = i.spec.allowedSourceRanges || []
o.spec.numberOfInstances = i.spec.numberOfInstances
o.spec.enableMasterLoadBalancer = i.spec.enableMasterLoadBalancer || false
o.spec.enableReplicaLoadBalancer = i.spec.enableReplicaLoadBalancer || false
o.spec.volume = { size: i.spec.volume.size }
if ('users' in i.spec && typeof i.spec.users === 'object') {
o.spec.users = Object.mapValues(i.spec.users, roleFlags =>
!Array.isArray(roleFlags)
? []
: roleFlags.filter(roleFlag => typeof roleFlag === 'string')
)
}
if ('databases' in i.spec && typeof i.spec.databases === 'object') {
o.spec.databases = Object.filterValues(i.spec.databases, owner =>
typeof owner === 'string'
)
}
if ('resources' in i.spec && typeof i.spec.resources === 'object') {
o.spec.resources = {};
['limits', 'requests'].forEach(section => {
const resources = i.spec.resources[section]
o.spec.resources[section] = {}
if (typeof resources === 'object') {
[
'cpu',
'memory',
].forEach(resourceType => {
if (resourceType in resources) {
const resourceClaim = resources[resourceType]
if (typeof resourceClaim === '') {
o.spec.resources[section][resourceType] = resources[resourceType]
}
}
})
}
})
}
this.editablePropertiesText = (
yamlParser
.safeDump(this.editableProperties)
.slice(0, -1)
)
this.editablePropertiesPreview = this.editablePropertiesText
this.update()
})
}
this.on('mount', () => {
const namespace = this.namespace = this.opts.namespace
const clustername = this.clustername = this.opts.clustername
const qname = this.qname = namespace + '/' + clustername
const cluster_path = this.cluster_path = (
encodeURI(namespace)
+ '/' + encodeURI(clustername)
)
this.progress = {
requestStatus: 'OK',
}
this.pollProgress()
})
this.on('updated', () => {
this.refs.yamlEditable.innerHTML = this.editablePropertiesPreview
Prism.highlightAll()
})

View File

@ -0,0 +1,29 @@
help-edit
h2 Help
.well
h3(style='margin-top: 0')
| Docs
a(href="{ opts.config.docs_link }")
| more...
h3 Editing
p.
The text box shows you the properties that can currently edit. Verify that the preview shows a valid part of the spec before submitting. After a successful submit the changes may take some time to be applied.
h3 Volume size
p.
You need to specify in format "123Gi". You can only increase the volume size. You can only increase volume size once very 6 hours, per AWS limitation.
virtual(
if='{ opts.config.static_network_whitelist && Object.keys(opts.config.static_network_whitelist).length > 0 }'
)
h3 IP Ranges
// Raw tags are required here, as otherwise either riotjs removes space it shouldn't, or pugjs adds space it shouldn't. And it has to be all in one line, as it has to be a pre tag (otherwise the riotjs compiler breaks the whitespace).
<pre><virtual each="{ network, network_index in Object.keys(opts.config.static_network_whitelist) }"><virtual if="{ network_index > 0 }"><br></virtual> # { network }<br><virtual each="{ range, range_index in opts.config.static_network_whitelist[network] }"> - { range }<virtual if="index < network.length - 1"><br></virtual></virtual></virtual></pre>

View File

@ -0,0 +1,18 @@
help-general
h2 Help
.well
h3(style='margin-top: 0')
| Docs
a(href="{ opts.config.docs_link }")
| more...
h3 Basics
p.
The PostgreSQL operator will use your definition to create a new
PostgreSQL cluster for you. You can either copy the yaml definition
to the repositiory or you can just hit create cluster.

81
ui/app/src/logs.tag.pug Normal file
View File

@ -0,0 +1,81 @@
logs
h1.page-header(if='{ cluster_path }')
nav(aria-label="breadcrumb")
ol.breadcrumb
li.breadcrumb-item
a(href='/#/list')
| PostgreSQL clusters
li.breadcrumb-item
a(href='/#/status/{ cluster_path }')
| { qname }
li.breadcrumb-item
a(href='/#/logs/{ cluster_path }')
| Logs
.sk-spinner-pulse(if='{ logs === undefined }')
.container-fluid(if='{ logs === null }')
p
| Error loading logs. Please
|
a(onclick="window.location.reload(true)") try again
|
| or
|
a(href="/") start over
| .
.container-fluid(if='{ logs }')
table.table.table-hover
tr(each='{ logs }')
td(each='{ [levels[Level]] }')
span.label.label-font-size(class='label-{ color_class }')
| { label }
td(style='white-space: pre')
| { Time }
td(style='font-family: monospace')
| { Message }
script.
this.levels = {
"panic": { label: "Panic" , color_class: "danger" },
"fatal": { label: "Fatal" , color_class: "danger" },
"error": { label: "Error" , color_class: "danger" },
"warning": { label: "Warning", color_class: "warning" },
"info": { label: "Info" , color_class: "primary" },
"debug": { label: "Debug" , color_class: "warning" },
}
this.logs = undefined
this.on('mount', () => {
if (
this.namespace !== this.opts.namespace
|| this.clustername !== this.opts.clustername
) {
const namespace = this.namespace = this.opts.namespace
const clustername = this.clustername = this.opts.clustername
const qname = this.qname = namespace + '/' + clustername
const cluster_path = this.cluster_path = (
encodeURI(namespace)
+ '/' + encodeURI(clustername)
)
;(
jQuery
.get(`/operator/clusters/${cluster_path}/logs`)
.done(logs => this.logs = logs.reverse())
.fail(() => this.logs = null)
.always(() => this.update())
)
}
})

937
ui/app/src/new.tag.pug Normal file
View File

@ -0,0 +1,937 @@
new
style.
.input-units {
width: 2em;
}
.resource-type {
text-align: left;
width: 7em;
}
.container-fluid
h1.page-header
nav(aria-label='breadcrumb')
ol.breadcrumb
li.breadcrumb-item
a(href='/#/new')
| New PostgreSQL cluster
.row.text-center(if='{ !creating }')
.row
.col-lg-3
h2 Cluster YAML definition
div
pre
code.language-yaml(ref='yamlNice')
.col-lg-6
form(
if='{ !creating }'
action='javascript:void(0);'
id='form'
)
.btn-toolbar.pull-right
.btn-group(
aria-label='Actions'
role='group'
)
input.btn.btn-primary(
type='submit'
form='form'
value='Validate'
)
button.btn.btn-info.btn-copy
| Copy definiton
button.btn.btn-success(
if='{ !clusterExists && parent.read_write }'
ref='submitbutton'
onclick='{ requestCreate }'
disabled='{ !allValid() }'
)
| Create cluster
a.btn.btn-small.btn-warning(
if='{ clusterExists }'
href='/#/status/{ namespace.state }/{ team }-{ name }'
)
| Cluster exists (show status)
h2 New cluster configuration
table.table
tr(
each='{ backup.state.type.state === "empty" ? [] : [backup] }'
)
td(colspan='2')
h3.text-center Source data
tr(
each='{ backup.state.type.state === "empty" ? [] : [backup] }'
)
td Backup name
td
input.form-control(
each='{ !["restore", "pitr"].includes(state.type.state) ? [] : [state.name] }'
ref='backup_name'
type='text'
placeholder='Source backup name'
required
value='{ state }'
onchange='{ edit }'
onkeyup='{ edit }'
)
tr(
each='{ backup.state.type.state === "empty" ? [] : [backup] }'
)
td Backup UID
td
input.form-control(
each='{ !["restore", "pitr"].includes(state.type.state) ? [] : [state.uid] }'
ref='backup_uid'
type='text'
placeholder='Source backup ID'
value='{ state }'
onchange='{ edit }'
onkeyup='{ edit }'
)
tr(
each='{ backup.state.type.state === "empty" ? [] : [backup] }'
)
td Target timestamp
td
input.timestamp.form-control(
each='{ !["pitr"].includes(state.type.state) ? [] : [state.timestamp] }'
ref='backup_timestamp'
type='text'
placeholder='Restore to state at timestamp'
value='{ state }'
onchange='{ edit }'
onkeyup='{ edit }'
)
tr(
each='{ backup.state.type.state === "empty" ? [] : [backup] }'
)
td(colspan='2')
h3.text-center New cluster settings
tr
td
| Name
td
input.form-control(
ref='name'
type='text'
placeholder='new-cluster (can be { 53 - team.length - 1 } characters long)'
title='Database cluster name, must be a valid hostname component'
pattern='[a-z0-9]+[a-z0-9\-]+[a-z0-9]+'
maxlength='{ 53 - team.length - 1 }'
required
value='{ name }'
onchange='{ nameChange }'
onkeyup='{ nameChange }'
style='width: 100%'
)
tr(
each='{ !["", "*"].includes(config.target_namespace) ? [] : [namespace] }'
)
td
| Namespace
td
select.form-control(
ref='namespace'
title='Database cluster Kubernetes namespace'
onchange='{ edit }'
onkeyup='{ edit }'
style='width: 100%'
)
option(
each='{ namespace in parent.config.namespaces }'
selected='{ state === namespace }'
value='{ namespace }'
)
| { namespace }
tr
td Owning team
td
select.form-control(
name='team'
onchange='{ teamChange }'
onkeyup='{ teamChange }'
)
option(
each='{ team in teams }'
value='{ team }'
)
| { team }
tr
td PostgreSQL version
td
select.form-control(
name='postgresqlVersion'
onchange='{ versionChange }'
onkeyup='{ versionChange }'
)
option(
each='{ version in opts.config.postgresql_versions }'
value='{ version }'
)
| { version }
tr
td DNS name:
td
| { dnsName }
tr
td Number of instances
td
input.form-control(
ref='instanceCount'
type='number'
min='1'
required
value='{ instanceCount }'
onchange='{ instanceCountChange }'
onkeyup='{ instanceCountChange }'
style='width: 100%'
)
tr(if='{ [undefined, true].includes(config.master_load_balancer_visible) }')
td Master load balancer
td
label
input(
type='checkbox'
value='{ enableMasterLoadBalancer }'
onchange='{ toggleEnableMasterLoadBalancer }'
)
|
| Enable master ELB
tr(if='{ [undefined, true].includes(config.replica_load_balancer_visible) }')
td Replica load balancer
td
label
input(
type='checkbox'
value='{ enableReplicaLoadBalancer }'
onchange='{ toggleEnableReplicaLoadBalancer }'
)
|
| Enable replica ELB
tr
td Volume size
td
.input-group
input.form-control(
ref='volumeSize'
type='number'
min='1'
required
value='{ volumeSize }'
onchange='{ volumeChange }'
onkeyup='{ volumeChange }'
)
.input-group-addon
.input-units Gi
tr(if='{ config.users_visible }')
td
button.btn.btn-success.btn-xs(onclick='{ users.add }')
span.glyphicon.glyphicon-plus
|
| Users
td
.input-group(each='{ users.state }')
.input-group-btn
button.btn.btn-danger(onclick='{ users.remove }')
span.glyphicon.glyphicon-trash
input.username.form-control(
type='text'
placeholder='Username'
title='^[a-z0-9]([-_a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-_a-z0-9]*[a-z0-9])?)*$'
pattern='^[a-z0-9]([-_a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-_a-z0-9]*[a-z0-9])?)*$'
required
value='{ state }'
onchange='{ edit }'
onkeyup='{ edit }'
style='width: 100%'
)
tr(if='{ config.databases_visible }')
td
button.btn.btn-success.btn-xs(onclick='{ databases.add }')
span.glyphicon.glyphicon-plus
|
| Databases
td
.input-group(each='{ databases.state }')
.input-group-btn
button.btn.btn-danger(
onclick='{ databases.remove }'
style='float: right'
)
span.glyphicon.glyphicon-trash
input.databasename.form-control(
type='text'
placeholder='Database name'
title='Alphanumerics or underscore; may not start on a digit'
pattern='^[a-zA-Z_][a-zA-Z0-9_]*$'
required
value='{ state }'
onchange='{ edit }'
onkeyup='{ edit }'
style='width: 50%'
)
select.owner.form-control(
onchange='{ owner.edit }'
onkeyup='{ owner.edit }'
disabled='{ users.valids.length === 0 }'
style='width: 50%'
)
option(
selected='{ owner.state.length === 0 }'
value=''
disabled
hidden
)
| Select owner…
option(
each='{ users.valids }'
selected='{ state === parent.owner.state }'
value='{ state }'
)
| { state }
tr(if='{ !jQuery.isEmptyObject(config.static_network_whitelist) }')
td Allowed IP ranges
td
ul.ips
li(each='{ value, name in config.static_network_whitelist }')
label
input(
type='checkbox'
value='{ name }'
onchange='{ toggleIPRange }'
checked='{ name in ranges }'
)
|
| { name }
tr(if='{ config.nat_gateways_visible }')
td
button.btn.btn-success.btn-xs(onclick='{ nats.add }')
span.glyphicon.glyphicon-plus
|
| AWS NAT IPs
td
div(
each='{ nats.state }'
)
.input-group
.input-group-btn
button.btn.btn-danger(onclick='{ nats.remove }')
span.glyphicon.glyphicon-trash
input.form-control(
type='text'
required
value='{ state }'
onchange='{ edit }'
onkeyup='{ edit }'
)
.input-group-addon
.input-units / 32
tr(if='{ config.odd_host_visible }')
td Odd host
td
.input-group
input.form-control(
ref='odd'
type='text'
name='odd'
placeholder='IP'
onchange='{ oddChanges }'
onkeyup='{ oddChanges }'
value='{ odd }'
)
.input-group-addon
.input-units / 32
tr(if='{ config.resources_visible }')
td Resources
td
table(width='100%')
tr
td CPU
td
.input-group
.input-group-addon.resource-type Request
input.form-control(
ref='cpuRequest'
type='number'
placeholder='{ cpu.state.request.initialValue }'
min='1'
required
value='{ cpu.state.request.state }'
onchange='{ cpu.state.request.edit }'
onkeyup='{ cpu.state.request.edit }'
)
.input-group-addon
.input-units m
.input-group
.input-group-addon.resource-type Limit
input.form-control(
ref='cpuLimit'
type='number'
placeholder='{ cpu.state.limit.initialValue }'
min='1'
required
value='{ cpu.state.limit.state }'
onchange='{ cpu.state.limit.edit }'
onkeyup='{ cpu.state.limit.edit }'
)
.input-group-addon
.input-units m
tr
td Memory
td
.input-group
.input-group-addon.resource-type Request
input.form-control(
ref='memoryRequest'
type='number'
placeholder='{ memory.state.request.initialValue }'
min='1'
required
value='{ memory.state.request.state }'
onchange='{ memory.state.request.edit }'
onkeyup='{ memory.state.request.edit }'
)
.input-group-addon
.input-units Gi
.input-group
.input-group-addon.resource-type Limit
input.form-control(
ref='memoryLimit'
type='number'
placeholder='{ memory.state.limit.initialValue }'
min='1'
required
value='{ memory.state.limit.state }'
onchange='{ memory.state.limit.edit }'
onkeyup='{ memory.state.limit.edit }'
)
.input-group-addon
.input-units Gi
.col-lg-3
help-general(config='{ opts.config }')
script.
// Pass a refresh callback for this tag to the options constructor argument
// used for all Dynamic objects built in this tag:
const add_refresh = object => Object.assign(
{},
object,
{ refresh: () => this.update() }
)
const Dynamic = options => this.opts.opts.Dynamic(add_refresh(options))
const Dynamics = options => this.opts.opts.Dynamics(add_refresh(options))
const DynamicSet = (items, options) => (
this.opts.opts.DynamicSet(items, add_refresh(options))
)
const dedent = require('dedent-js')
// Dedent twice because pug adds tabs:
this.yamlTemplate = dedent`
kind: "postgresql"
apiVersion: "acid.zalan.do/v1"
metadata:
name: "{{ team }}-{{ name }}"
namespace: "{{ namespace.state }}"
labels:
team: {{ team }}
spec:
teamId: "{{ team }}"
postgresql:
version: "{{ postgresqlVersion }}"
numberOfInstances: {{ instanceCount }}
{{#if enableMasterLoadBalancer}}
enableMasterLoadBalancer: true
{{/if}}
{{#if enableReplicaLoadBalancer}}
enableReplicaLoadBalancer: true
{{/if}}
volume:
size: "{{ volumeSize }}Gi"
{{#if users}}
users:{{#each users}}
{{ state }}: []{{/each}}{{/if}}
{{#if databases}}
databases:{{#each databases}}
{{ state }}: {{ owner.state }}{{/each}}{{/if}}
allowedSourceRanges:
# IP ranges to access your cluster go here
{{#each ranges}} # {{ @key }}
{{#each this }}
- {{ this }}
{{/each}}
{{/each}}{{#if nats}}
# NAT instances
{{#each nats}}
- {{ state }}/32
{{/each}}{{/if}}{{#if odd}}
# Your odd host IP
- {{ odd }}/32
{{/if}}
resources:
requests:
cpu: {{ cpu.state.request.state }}m
memory: {{ memory.state.request.state }}Gi
limits:
cpu: {{ cpu.state.limit.state }}m
memory: {{ memory.state.limit.state }}Gi{{#if restoring}}
clone:
cluster: "{{ backup.state.name.state }}"
uid: "{{ backup.state.uid.state }}"{{#if pitr}}
timestamp: "{{ backup.state.timestamp.state }}"{{/if}}{{/if}}
`
const yamlParser = require('js-yaml')
this.teams = this.opts.teams
this.config = this.opts.config
this.getContext = () => {
return {
name: this.name.toLowerCase(),
team: this.team.toLowerCase(),
postgresqlVersion: this.postgresqlVersion,
instanceCount: this.instanceCount,
enableMasterLoadBalancer: this.enableMasterLoadBalancer,
enableReplicaLoadBalancer: this.enableReplicaLoadBalancer,
volumeSize: this.volumeSize,
users: this.users.valids,
databases: this.databases.valids,
ranges: this.ranges,
nats: this.nats.valids,
odd: this.odd,
cpu: this.cpu,
memory: this.memory,
backup: this.backup,
namespace: this.namespace,
restoring: this.backup.state.type.state !== 'empty',
pitr: this.backup.state.type.state === 'pitr',
}
}
this.getYAML = () => {
yaml = Handlebars.compile(this.yamlTemplate)
yaml = yaml(this.getContext())
return yaml
}
this.on('update', () => {
this.teams = this.opts.teams
this.config = this.opts.config
yaml = Handlebars.compile(this.yamlTemplate)
yaml = yaml(this.getContext())
this.refs.yamlNice.innerHTML = yaml
Prism.highlightAll()
})
this.oddChanges = e => {
this.odd = e.target.value
}
this.ranges = {}
this.toggleIPRange = e => {
if (e.target.checked) {
this.ranges[e.target.value] = this.config.static_network_whitelist[e.target.value]
} else {
delete this.ranges[e.target.value]
}
this.update()
}
this.toggleEnableMasterLoadBalancer = e => {
this.enableMasterLoadBalancer = !this.enableMasterLoadBalancer
}
this.toggleEnableReplicaLoadBalancer = e => {
this.enableReplicaLoadBalancer = !this.enableReplicaLoadBalancer
}
this.volumeChange = e => {
this.volumeSize = +e.target.value
}
this.updateDNSName = () => {
this.dnsName = this.config.dns_format_string.format(
this.name,
this.team,
this.namespace.state,
)
}
this.updateClusterName = () => {
this.clusterName = (this.team + '-' + this.name).toLowerCase()
this.checkClusterExists()
this.updateDNSName()
}
this.nameChange = e => {
this.name = e.target.value.toLowerCase()
this.updateClusterName()
}
this.teamChange = e => {
this.team = e.target.value.toLowerCase()
this.updateClusterName()
}
this.instanceCountChange = e => {
this.instanceCount = +e.target.value
}
this.checkClusterExists = () => (
jQuery
.get(
'/postgresqls/'
+ this.namespace.state
+ '/'
+ this.clusterName
)
.done(() => this.clusterExists = true)
.fail(() => this.clusterExists = false)
.always(() => this.update())
)
this.versionChange = e => {
this.postgresqlVersion = e.target.value
}
this.requestCreate = e => {
jsonPayload = JSON.stringify(yamlParser.safeLoad(this.getYAML()))
this.creating = true
this.update()
jQuery.ajax({
type: 'POST',
url: '/create-cluster',
contentType:'application/json',
data: jsonPayload,
processData: false,
success: () => {
route(
'/status/'
+ encodeURI(this.namespace.state)
+ '/'
+ encodeURI(this.clusterName)
)
},
error: (r, status, error) => {
console.log('Create request failed')
},
dataType: 'json'
})
}
const clipboard = new Clipboard('.btn-copy', {
text: () => { return this.getYAML() }
})
this.namespace = Dynamic({
init: () => (
!this.config.target_namespace
|| ['', '*'].includes(this.config.target_namespace)
? 'default'
: this.config.target_namespace
),
})
;{
const update_namespace = this.namespace.update
this.namespace.update = value => {
update_namespace(value)
this.updateClusterName()
this.update()
}
}
this.nats = Dynamics()
this.users = Dynamics()
this.users.itemInit = () => Dynamic({
update: (value, instance, event) => {
if (value === '' || instance.state.length > 0) {
this.databases.state.forEach(database => {
if (database.owner.state === instance.state) {
database.owner.update(value, database.owner, event)
}
})
}
instance.state = value
this.update()
// Note: there is a bug here somewhere.
//
// To reproduce it:
// 1. Add user user1
// 2. Add user user2
// 3. Add user user22
// 4. Add database 1
// 5. Add database 2
// 6. Set database 1 owner to user2
// 7. Set database 2 owner to user22
// 8. Type another 2 at the end of the username for user2, so that it becomes a repeated copy of user22. It should automatically set the database 1 owner to user22.
// 9. Delete the character you just typed, so that the first user22, which used to be user2 before the last step, becomes user2 again.
//
// In principle, that last step should set the owners for both database 1 and database 2 to user2, as both databases had owner user22 which was edited. Instead, the owner for database 1 becomes user1 (WTF???) and the owner for database 2 becomes user2 (which is correct). Note that the tag states are updated correctly, and the HTML shown in the Chrome element inspector has the selected attribute in the correct option elements — yet the wrong owner option is selected in the rendered select control on the page for the owner of database 1.
//
// Solution attempted that failed because riotjs does weird things with the DOM:
//
// $('select.owner', this.root).each(select => {
// var options = $(select).children()
// for (var i = 0; i < options.length; i++) {
// const option = options[i]
// const selected = $(option).attr('selected')
// if (selected === 'true' || selected === 'selected') {
// select.selectedIndex = i
// child.selected = true
// } else {
// child.selected = false
// }
// }
// })
//
// Hours wasted on this stupid bug: 3
// Please keep this counter updated.
}
})
{
const baseRemoveUser = this.users.remove
this.users.remove = event => {
this.databases.state.forEach(database => {
if (database.owner.state === event.item.state) {
database.owner.update('', database.owner, event)
}
})
baseRemoveUser(event)
this.update()
}
}
this.databases = Dynamics({
itemInit: () => {
const item = Dynamic()
const baseItemUpdate = item.update
item.update = (value, instance, event) => {
this.check()
baseItemUpdate(value, instance, event)
}
item.owner = Dynamic({
validState: username => (
this.users.valids.find(user => user.state === username) !== undefined
)
})
return item
},
itemValid: item => item.valid() && item.owner.valid(),
})
const DynamicResource = options => {
const instance = DynamicSet({
request: () => options.request,
limit: () => options.limit,
})
instance.state.request.initialValue = options.request
instance.state.limit.initialValue = options.limit
return instance
}
this.cpu = DynamicResource({ request: 100, limit: 1000 })
this.memory = DynamicResource({ request: 1, limit: 1 })
this.backup = DynamicSet({
type: () => 'empty',
name: () => '',
uid: () => '',
timestamp: () => '',
})
const pitr_timestamp_format = 'YYYY-MM-DDTHH:mm:ss.SSSZ'
this.backup.state.timestamp.validState = (
base => (
state => {
if (!base(state)) {
return false
}
const parsed = moment.utc(state, pitr_timestamp_format, true)
return moment.isMoment(parsed) && !isNaN(parsed.utcOffset())
}
)
)(this.backup.state.timestamp.validState)
this.allValid = () => (
!this.creating &&
this.refs.name && this.refs.name.validity.valid &&
this.refs.instanceCount && this.refs.instanceCount.validity.valid && Number.isInteger(+this.instanceCount) &&
this.refs.volumeSize && this.refs.volumeSize.validity.valid && Number.isInteger(+this.volumeSize) &&
this.refs.cpuRequest && this.refs.cpuRequest.validity.valid && Number.isInteger(+this.cpu.state.request.state) &&
this.refs.cpuLimit && this.refs.cpuLimit.validity.valid && Number.isInteger(+this.cpu.state.limit.state) &&
this.refs.memoryRequest && this.refs.memoryRequest.validity.valid && Number.isInteger(+this.memory.state.request.state) &&
this.refs.memoryLimit && this.refs.memoryLimit.validity.valid && Number.isInteger(+this.memory.state.limit.state) &&
[
this.users,
this.databases,
this.backup,
].every(x => x.valid())
)
this.check = () => {
const setValidity = validity => (_, element) => (
element.setCustomValidity(
validity(element)
)
)
$('select.owner', this.root).each(setValidity(owner =>
owner.value ? '' : 'Must select an owner for this database'
))
const counts = (values, key=x => x) => values.map(key).reduce(
(o, item) => {
if (item in o) { o[item] += 1 }
else { o[item] = 1 }
return o
},
{}
)
$('input.timestamp', this.root).each(setValidity(timestamp =>
this.backup.state.timestamp.valid() ? '' :
'Timestamp must be formatted as ISO-8601 with milliseconds; note that a UTC offset as ±HH:mm is required: YYYY-MM-DDTHH:mm:ss.SSSZ e.g. 2018-12-31T23:59:59.999+14:00'
))
const userCounts = counts(this.users.state, item => item.state)
$('input.username', this.root).each(setValidity(name =>
!name.value || userCounts[name.value] === 1 ? '' :
'Usernames must not repeat'
))
const databaseCounts = counts(this.databases.state, item => item.state)
$('input.databasename', this.root).each(setValidity(name =>
!name.value || databaseCounts[name.value] === 1 ? '' :
'Database names must not repeat'
))
if (this.refs.submitbutton) {
this.refs.submitbutton.disabled = (
$(document.querySelectorAll(':invalid')).length !== 0
) || $('select.owner:disabled').length !== 0
}
}
this.on('updated', this.check)
this.reset_form = () => {
if (this.pollProgressTimer) {
clearInterval(this.pollProgressTimer)
}
this.creating = false
this.name = ''
if (this.teams && this.teams.length > 0) {
this.team = this.teams[0]
} else {
this.team = ''
}
this.clusterName = (this.name + '-' + this.team).toLowerCase()
this.volumeSize = 10
this.instanceCount = 1
this.ranges = {}
this.odd = ''
this.enableMasterLoadBalancer = false
this.enableReplicaLoadBalancer = false
this.postgresqlVersion = this.postgresqlVersion = (
this.config.postgresql_versions[0]
)
this.updateDNSName();
this.check()
this.backup.state.type.update(
this.opts.backup_name === undefined
? 'empty'
: this.opts.backup_timestamp === undefined
? 'restore'
: 'pitr'
)
this.backup.state.name.update(
decodeURI(this.opts.backup_name || '')
)
this.backup.state.uid.update(
!this.opts.backup_uid || this.opts.backup_uid === 'base'
? ''
: this.opts.backup_uid
)
this.backup.state.timestamp.update(
!this.opts.backup_timestamp
? ''
: moment.utc(
decodeURI(this.opts.backup_timestamp)
).format(pitr_timestamp_format)
)
this.opts && this.opts.backup_name && delete this.opts.backup_name
this.opts && this.opts.backup_uid && delete this.opts.backup_uid
this.opts && this.opts.backup_timestamp && delete this.opts.backup_timestamp
this.update()
}
this.on('mount', this.reset_form)

View File

@ -0,0 +1,219 @@
postgresql
.container-fluid
h1.page-header
nav(aria-label='breadcrumb')
ol.breadcrumb
li.breadcrumb-item
a(href='/#/list')
| PostgreSQL clusters
li.breadcrumb-item(if='{ cluster_path }')
a(href='/#/status/{ cluster_path }')
| { qname }
.row(if='{ cluster_path }')
.col-lg-3
h2 Cluster YAML definition
div(if='{ progress.postgresql }')
<pre><code ref="yamlNice" class="language-yaml"></code></pre>
pre(if='{ !progress.postgresql }')
code # Loading
virtual(if='{ uid }')
h3 Cluster UID
code { uid }
.col-lg-6
div
.btn-toolbar.pull-right
.btn-group(
aria-label='Actions'
role='group'
)
a.btn.btn-info(
href='/#/logs/{ cluster_path }'
)
| Logs
a.btn(
class='btn-{ opts.read_write ? "primary" : "info" }'
if='{ progress.postgresql }'
href='/#/clone/{ clustername }/{ uid }/{ encodeURI(new Date().toISOString()) }'
)
| Clone
a.btn(
class='btn-{ opts.read_write ? "warning" : "info" }'
href='/#/edit/{ cluster_path }'
)
| Edit
button.btn.btn-danger(
if='{ opts.read_write }'
onclick='{ delete_cluster }'
)
| Delete
h2 Checking status of cluster
.progress
.progress-bar.progress-bar-success(style='width: 20%' if='{ progress.requestStatus === "OK" || progress.masterLabel }')
.progress-bar.progress-bar-success(style='width: 20%' if='{ progress.postgresql }')
.progress-bar.progress-bar-success(style='width: 20%' if='{ progress.statefulSet }')
.progress-bar.progress-bar-success(style='width: 20%' if='{ progress.containerFirst }')
.progress-bar.progress-bar-success(style='width: 20%' if='{ progress.masterLabel }')
.progress-bar.progress-bar-info.progress-bar-striped.active(if='{ !progress.masterLabel }' style='width: 10%')
.alert.alert-info(if='{ !progress.requestStatus }') PostgreSQL cluster requested
.alert.alert-danger(if='{ progress.requestStatus !== "OK" }') Create request failed
.alert.alert-success(if='{ progress.requestStatus === "OK" }') Create request successful ({ new Date(progress.createdTimestamp).toLocaleString() })
.alert.alert-info(if='{ !progress.statefulSet }') StatefulSet pending
.alert.alert-success(if='{ progress.statefulSet }') StatefulSet created
.alert.alert-info(if='{ progress.statefulSet && !progress.containerFirst }') Waiting for 1st container to spawn
.alert.alert-success(if='{ progress.containerFirst }') First PostgreSQL cluster container spawned
.alert.alert-info(if='{ !progress.postgresql }') PostgreSQL cluster manifest pending
.alert.alert-success(if='{ progress.postgresql }') PostgreSQL cluster manifest created
.alert.alert-info(if='{ progress.containerFirst && !progress.masterLabel }') Waiting for master to become available
.alert.alert-success(if='{ progress.masterLabel }') PostgreSQL master available, label is attached
.alert.alert-success(if='{ progress.masterLabel && progress.dnsName }') PostgreSQL ready: <strong>{ progress.dnsName }</strong>
.col-lg-3
help-general(config='{ opts.config }')
script.
var yamlParser = require('js-yaml')
this.delete_cluster = _ => this.parent.opts.delete_cluster(
this.namespace,
this.clustername,
)
this.progress = {}
this.progress.requestStatus = 'OK'
this.pollProgressTimer = false
this.startPollProgress = () => {
this.pollProgressTimer = setInterval(this.pollProgress, 10000)
}
this.stopPollProgress = () => {
clearInterval(this.pollProgressTimer)
this.pollProgressTimer = false
}
this.pollProgress = () => {
jQuery.get(
'/postgresqls/' + this.cluster_path,
).done(data => {
this.progress.postgresql = true
this.progress.postgresqlManifest = data
this.progress.createdTimestamp = data.metadata.creationTimestamp
this.uid = this.progress.postgresqlManifest.metadata.uid
this.update()
jQuery.get(
'/statefulsets/' + this.cluster_path,
).done(data => {
this.progress.statefulSet = true
this.update()
jQuery.get(
'/statefulsets/' + this.cluster_path + '/pods',
).done(data => {
if (data.length > 0) {
this.progress.containerFirst = true
}
masters = data.filter((x) => { return x.labels['spilo-role'] === 'master'} )
if (masters.length === 1) {
this.progress.masterLabel = true
}
this.update()
jQuery.get(
'/services/' + this.cluster_path,
).done(data => {
if (data.metadata && data.metadata.annotations && 'zalando.org/dnsname' in data.metadata.annotations) {
this.progress.dnsName = data.metadata.annotations['zalando.org/dnsname']
} else if (data.metadata && data.metadata.annotations && 'external-dns.alpha.kubernetes.io/hostname' in data.metadata.annotations) {
this.progress.dnsName = data.metadata.annotations['external-dns.alpha.kubernetes.io/hostname']
} else {
// Kubernetes Service name should resolve
this.progress.dnsName = data.metadata.name + '.' + data.metadata.namespace
}
this.update()
})
})
})
})
}
this.on('mount', () => {
this.uid = undefined
const namespace = this.namespace = this.opts.namespace
const clustername = this.clustername = this.opts.clustername
const qname = this.qname = namespace + '/' + clustername
const cluster_path = this.cluster_path = (
encodeURI(namespace)
+ '/' + encodeURI(clustername)
)
this.stopPollProgress()
this.pollProgress()
this.startPollProgress()
})
this.on('unmount', () =>
this.stopPollProgress()
)
this.on('update', () => {
if (this.progress.postgresqlManifest) {
const manifest = this.progress.postgresqlManifest
const last_applied = 'kubectl.kubernetes.io/last-applied-configuration'
if (manifest.metadata.annotations) {
delete manifest.metadata.annotations[last_applied]
}
delete manifest.metadata.creationTimestamp
delete manifest.metadata.deletionGracePeriodSeconds
delete manifest.metadata.deletionTimestamp
delete manifest.metadata.generation
delete manifest.metadata.resourceVersion
delete manifest.metadata.selfLink
delete manifest.metadata.uid
delete manifest.status
if (this.refs.yamlNice) {
this.refs.yamlNice.innerHTML = yamlParser.safeDump(
this.progress.postgresqlManifest,
{
sortKeys: true,
},
)
}
} else {
if(this.refs.yamlNice) {
this.refs.yamlNice.innerHTML = '# Loading postgresql cluster manifest'
}
}
Prism.highlightAll()
})

View File

@ -0,0 +1,247 @@
postgresqls
.container-fluid
h1.page-header
nav(aria-label='breadcrumb')
ol.breadcrumb
li.breadcrumb-item
a(href='/#/list')
| PostgreSQL clusters
.sk-spinner-pulse(
if='{ my_clusters !== null && other_clusters !== null && (my_clusters === undefined || other_clusters === undefined) }'
)
p(if='{ my_clusters === null || other_clusters === null }')
| Error loading clusters. Please
|
a(onclick='window.location.reload(true)') try again
|
| or
|
a(href='/') start over
| .
div(
if='{ my_clusters && other_clusters }'
)
p
| Search:
|
input(
type='text'
onchange='{ filter.edit }'
onkeyup='{ filter.edit }'
value='{ filter.state }'
)
.page-header
h1 My team's clusters ({ my_clusters.length })
table.table.table-hover(if='{ my_clusters.length > 0 }')
thead
tr
th(style='width: 120px') Team
th(style='width: 50px') Pods
th(style='width: 140px') CPU
th(style='width: 130px') Memory
th(style='width: 100px') Size
th(style='width: 130px') Namespace
th Name
tbody
tr(
each='{ my_clusters }'
hidden='{ !namespaced_name.toLowerCase().includes(filter.state.toLowerCase()) }'
)
td { team }
td { nodes }
td { cpu } / { cpu_limit }
td { memory } / { memory_limit }
td { volume_size }
td(style='white-space: pre')
| { namespace }
td
a(
href='/#/status/{ cluster_path(this) }'
)
| { name }
.btn-group.pull-right(
aria-label='Cluster { qname } actions'
role='group'
style='display: flex'
)
a.btn.btn-info(
href='/#/status/{ cluster_path(this) }'
)
i.fa.fa-check-circle.regular
| Status
a.btn.btn-info(
if='{ opts.config.pgview_link }'
href='{ opts.config.pgview_link }{ cluster_path(this) }'
)
i.fa.fa-chart-line
| Pgview
a.btn.btn-info(
href='/#/logs/{ cluster_path(this) }'
)
i.fa.fa-align-justify
| Logs
a.btn(
class='btn-{ opts.read_write ? "primary" : "info" }'
href='/#/clone/{ encodeURI(name) }/{ encodeURI(uid) }/{ encodeURI(new Date().toISOString()) }'
)
i.fa.fa-clone.regular
| Clone
a.btn(
class='btn-{ opts.read_write ? "warning" : "info" }'
href='/#/edit/{ cluster_path(this) }'
)
| Edit
button.btn.btn-danger(
if='{ opts.read_write }'
onclick='{ delete_cluster }'
)
| Delete
.page-header
h1 Other clusters ({ other_clusters.length})
table.table.table-hover(if='{ other_clusters.length > 0 }')
thead
tr
th(style='width: 120px') Team
th(style='width: 50px') Pods
th(style='width: 140px') CPU
th(style='width: 130px') Memory
th(style='width: 100px') Size
th(style='width: 130px') Namespace
th Name
tbody
tr(
each='{ other_clusters }'
hidden='{ !namespaced_name.toLowerCase().includes(filter.state.toLowerCase()) }'
)
td { team }
td { nodes }
td { cpu } / { cpu_limit }
td { memory } / { memory_limit }
td { volume_size }
td(style='white-space: pre')
| { namespace }
td
a(
href='/#/status/{ cluster_path(this) }'
)
| { name }
.btn-group.pull-right(
aria-label='Cluster { qname } actions'
role='group'
style='display: flex'
)
a.btn.btn-info(
href='/#/status/{ cluster_path(this) }'
)
i.fa.fa-check-circle.regular
| Status
a.btn.btn-info(
if='{ opts.config.pgview_link }'
href='{ opts.config.pgview_link }{ cluster_path(this) }'
target='_blank'
)
i.fa.fa-chart-line
| Pgview
a.btn.btn-info(
href='/#/logs/{ cluster_path(this) }'
)
i.fa.fa-align-justify
| Logs
a.btn(
class='btn-{ opts.read_write ? "primary" : "info" }'
href='/#/clone/{ encodeURI(name) }/{ encodeURI(uid) }/{ encodeURI(new Date().toISOString()) }'
)
i.fa.fa-clone.regular
| Clone
a.btn(
class='btn-{ opts.read_write ? "warning" : "info" }'
href='/#/edit/{ cluster_path(this) }'
)
| Edit
button.btn.btn-danger(
if='{ opts.read_write }'
onclick='{ delete_cluster }'
)
| Delete
script.
// Pass a refresh callback for this tag to the options constructor argument
// used for all Dynamic objects built in this tag:
const add_refresh = object => Object.assign(
{},
object,
{ refresh: () => this.update() }
)
const Dynamic = options => this.parent.opts.Dynamic(add_refresh(options))
this.filter = Dynamic()
this.my_clusters = undefined
this.other_clusters = undefined
this.delete_cluster = event => this.parent.opts.delete_cluster(
event.item.namespace,
event.item.name,
)
const cluster_path = this.cluster_path = cluster => (
encodeURI(cluster.namespace)
+ '/' + encodeURI(cluster.name)
)
this.on('mount', () =>
jQuery
.get('/postgresqls')
.done(clusters => {
this.my_clusters = []
this.other_clusters = []
clusters.forEach(cluster =>
(
this.opts.teams.includes(
cluster.team.toLowerCase()
)
? this.my_clusters
: this.other_clusters
).push(cluster)
)
})
.fail(() => {
this.my_clusters = null
this.other_clusters = null
})
.always(() => this.update())
)

3
ui/app/src/prism.js Normal file

File diff suppressed because one or more lines are too long

472
ui/app/src/restore.tag.pug Normal file
View File

@ -0,0 +1,472 @@
restore
.container-fluid
.sk-spinner-pulse(if='{ stored_clusters === undefined }')
p(if='{ stored_clusters === null }')
| Error loading stored clusters. Please
|
a(onclick="window.location.reload(true);") try again
|
| or
|
a(href="/") start over
| .
p(if='{ stored_clusters && stored_clusters.length === 0 }')
| No stored clusters found.
div(if='{ stored_clusters && stored_clusters.length > 0 }')
h1.page-header
nav(aria-label='breadcrumb')
ol.breadcrumb
li.breadcrumb-item
a(href='/#/backups')
| PostgreSQL cluster backups ({ stored_clusters.length })
p
| Search:
|
input(
each='{ [filter] }'
type='text'
onchange='{ edit }'
onkeyup='{ edit }'
value='{ state }'
)
.stored-clusters.panel-group.collapsible
.stored-clusters.panel.panel-default.collapsible(
each='{ stored_clusters }'
hidden='{ !name.includes(filter.state) }'
)
.stored-clusters.panel-heading.collapsible(
id='{ id }-head'
class='{ collapsed ? "collapsed" : "" }'
data-toggle='collapse'
data-target='#{ id }-collapse'
)
a.panel-title.collapsible
| { name }
.stored-clusters.panel-collapse.collapse.collapsible(
id='{ id }-collapse'
data-stored-clusters='{ name }'
)
.stored-clusters.panel-body.collapsible(id='{ id }-body')
.sk-spinner-pulse(if='{ versions === undefined }')
p(if='{ versions === null }')
| Error loading backups. Please try again or
|
a(href="/") start over
| .
p(if='{ versions && versions.length === 0 }')
| No backups found.
div(if='{ versions && versions.length > 0 }')
.versions.panel-group.collapsible(style='margin-top: 0.3em')
.versions.panel.panel-default.collapsible(each='{ versions }')
.versions.panel-heading.collapsible(
id='{ id }-head'
class='{ collapsed ? "collapsed" : "" }'
data-toggle='collapse'
data-target='#{ id }-collapse'
)
a.versions.panel-title.collapsible
| { name }
.versions.panel-collapse.collapse.collapsible(
id='{ id }-collapse'
data-stored-clusters='{ parent.name }'
data-versions='{ name }'
)
.versions.panel-body.collapsible(id='{ id }-body')
.sk-spinner-pulse(if='{ basebackups === undefined }')
p(if='{ basebackups === null }')
| Error loading snapshots. Please try again or
|
a(href="/") start over
| .
p(if='{ basebackups && basebackups.length === 0 }')
| No snapshots found.
div(if='{ basebackups && basebackups.length > 0 }')
div(
style='margin-bottom: 0.3em'
)
.pull-left(style='margin-right: 0.3em')
button.btn.btn-primary.pull-left(
id='{ id }-clone'
)
| Clone at latest state
div
.input-group
.input-group-btn
button.btn.btn-info(id='{ id }-pitr')
| Clone at
.input.form-control(type='text')
| { clone_time && to_clone_time(clone_time) }
.timeline(id='{ id }-timeline')
.basebackups.panel-group.collapsible(style='margin-top: 0.3em')
.basebackups.panel.panel-default.collapsible(
each='{ basebackups }'
)
.basebackups.panel-heading.collapsible(
id='{ id }-head'
class='{ collapsed ? "collapsed" : "" }'
data-toggle='collapse'
data-target='#{ id }-collapse'
)
a.basebackups.panel-title.collapsible
| { relative_time(last_modified) }
span.label.label-success(
if='{ index === basebackups.length - 1 }'
style='margin-left: 0.5em'
)
| latest snapshot
.basebackups.panel-collapse.collapse.collapsible(
id='{ id }-collapse'
data-stored-clusters='{ parent.parent.name }'
data-versions='{ parent.name }'
data-backups='{ name }'
)
.basebackups.panel-body.collapsible(id='{ id }-body')
table.basebackups.table
tr
th Name
td { name }
tr
th Size
td { expanded_size_bytes }
tr
th Last modified
td { relative_time(last_modified) }
tr
th WAL segment backup start
td { wal_segment_backup_start }
tr
th WAL segment backup stop
td { wal_segment_backup_stop }
tr
th WAL segment offset backup start
td { wal_segment_offset_backup_start }
tr
th WAL segment offset backup stop
td { wal_segment_offset_backup_stop }
button.btn.btn-info(
id='{ id }-restore'
)
| Clone at this snapshot
script.
const sort_by = require('sort-by')
// Pass a refresh callback for this tag to the options constructor argument
// used for all Dynamic objects built in this tag:
const add_refresh = object => Object.assign(
{},
object,
{ refresh: () => this.update() }
)
const Dynamic = options => this.parent.opts.Dynamic(add_refresh(options))
const filter = this.filter = Dynamic()
const to_timestamp = this.to_timestamp = time => (
time
.replace('T', ' ')
.replace('.000Z', '')
)
const trunc_timestamp = this.trunc_timestamp = time => (
Math.trunc(time / 1000) * 1000
)
const to_clone_time = this.to_clone_time = time => to_timestamp(
new Date(trunc_timestamp(time))
.toISOString()
)
const min = (a, b) => a <= b ? a : b
const max = (a, b) => a >= b ? a : b
const minimum = array => array.reduce(min)
const maximum = array => array.reduce(max)
const both = (f, g, x) => [f(x), g(x)]
const setting = property => (object, value) => {
object[property] = value
return object
}
const load_time = this.load_time = +new Date()
const relative_time = this.relative_time = time => {
const relative = humanizeDuration(
trunc_timestamp(load_time)
- Date.parse(time)
)
return `${to_timestamp(time)} UTC (${relative} ago)`
}
const q = selector => jQuery(selector, this.root)
const route_on_click = (selector, target) => (
q(selector).on('click', event =>
route(target(event).join('/'))
)
)
const on_collapse = action => (
({
klass,
collection,
predicate = () => true,
selector_prefix = '',
body,
}) => (
q(`${selector_prefix}.${klass}.collapse`)
.on(
action + '.bs.collapse',
event => {
if (!(
event.target.classList.contains(klass)
&& predicate(event.target.dataset)
)) {
return true
}
const target = collection.find(item =>
item.name === event.target.getAttribute('data-' + klass)
)
target['collapsed'] = action === 'hide'
body(target)
this.update()
},
)
)
)
const collapsible_handlers = options => {
on_collapse('hide')(Object.assign({}, options, { body: _ => {}}))
on_collapse('show')(options)
}
const get_subresources_once = ({
parent_resource,
key,
url,
body,
build_subresource = subresource => subresource,
build_subresources = subresources => subresources,
}) => {
if (parent_resource[key] === undefined) {
(
jQuery
.get(url)
.done(values => {
parent_resource[key] = (
build_subresources(values)
.map(build_subresource)
)
this.update()
body(parent_resource[key])
})
.fail(() => parent_resource[key] = null)
.always(() => this.update())
)
}
}
this.stored_clusters = undefined
this.on('mount', () =>
get_subresources_once({
parent_resource: this,
key: 'stored_clusters',
url: '/stored_clusters',
build_subresource: stored_cluster_name => ({
id: 'stored-cluster-' + stored_cluster_name,
name: stored_cluster_name,
collapsed: true,
}),
body: stored_clusters => collapsible_handlers({
klass: 'stored-clusters',
collection: stored_clusters,
body: stored_cluster => get_subresources_once({
parent_resource: stored_cluster,
key: 'versions',
url: '/stored_clusters/' + stored_cluster.name,
build_subresource: version_name => ({
id: stored_cluster.id + '-version-' + version_name,
name: version_name,
collapsed: true,
stored_cluster: stored_cluster,
}),
body: versions => collapsible_handlers({
klass: 'versions',
collection: versions,
selector_prefix: `#${stored_cluster.id}-collapse `,
predicate: data => stored_cluster.name === data.storedClusters,
body: version => get_subresources_once({
parent_resource: version,
key: 'basebackups',
url: (
'/stored_clusters/' + stored_cluster.name
+ '/' + version.name
),
build_subresource: basebackup => Object.assign(basebackup, {
id: version.id + '-basebackup-' + basebackup.name,
}),
build_subresources: basebackups => (
basebackups
.sort(sort_by('last_modified'))
.map(setting('index'))
),
body: basebackups => {
if (basebackups.length === 0) {
return
}
const basebackup_age = basebackup => (
+new Date(basebackup.last_modified)
)
const oldest = version.basebackups[0]
const newest = version.basebackups[
version.basebackups.length - 1
]
const [start, end] = both(
maximum,
minimum,
[
load_time,
...[oldest, newest].map(basebackup_age),
],
)
const span = end - start
const padding_time = 0.1 * span
basebackups.forEach(basebackup => {
route_on_click(
'#' + basebackup.id + '-restore',
() => [
'/clone',
encodeURI(stored_cluster.name),
encodeURI(version.name),
// TODO: Ideally, this should use the basebackup's end
// LSN, not the S3 last modification timestamp. However,
// the current implementation of the clone feature does
// not allow specifying `recovery_target_lsn`.
encodeURI(basebackup.last_modified),
],
)
})
version.timeline = new vis.Timeline(
q('#' + version.id + '-timeline')[0],
new vis.DataSet([
...version.basebackups.map(
basebackup => ({
id: basebackup.index,
content: (
to_timestamp(basebackup.last_modified)
.replace(' ', '<br>')
+ ' (UTC)'
),
start: basebackup.last_modified,
})
),
{
id: version.basebackups.length,
content: 'now',
start: load_time,
type: 'point',
}
]),
{
min: min - padding_time,
max: max + 5 * padding_time,
moment: time => vis.moment(time).utc(),
clickToUse: true,
moveable: true,
zoomable: true,
showCurrentTime: true,
}
)
version.clone_time = trunc_timestamp(end - span / 3)
version.timeline.addCustomTime(
version.clone_time,
'clone_time',
)
version.timeline.on('timechange', properties => {
version.clone_time = +properties.time
this.update()
})
version.timeline.on('select', selection =>
version.basebackups.forEach(basebackup =>
q('#' + basebackup.id + '-collapse').collapse(
selection.items.includes(basebackup.index)
? 'show'
: 'hide'
)
)
)
route_on_click(
'#' + version.id + '-clone',
() => [
'/clone',
encodeURI(stored_cluster.name),
encodeURI(version.name),
encodeURI(to_clone_time(load_time)),
],
)
route_on_click(
'#' + version.id + '-pitr',
() => [
'/clone',
encodeURI(stored_cluster.name),
encodeURI(version.name),
encodeURI(to_clone_time(version.clone_time)),
],
)
},
}),
}),
}),
}),
})
)

149
ui/app/src/status.tag.pug Normal file
View File

@ -0,0 +1,149 @@
status
.container-fluid
h1.page-header
nav(aria-label="breadcrumb")
ol.breadcrumb
li.breadcrumb-item
a(href='/#/operator')
| Workers
virtual(if='{ operatorShowLogs && logs }')
li.breadcrumb-item { worker_id }
li.breadcrumb-item
a(href='/#/operator/worker/{ worker_id }/logs')
| Logs
virtual(if='{ operatorShowQueue && queue }')
li.breadcrumb-item { worker_id }
li.breadcrumb-item
a(href='/#/operator/worker/{ worker_id }/queue')
| Queue
div(if='{ status }')
div(if='{ !operatorShowLogs && !operatorShowQueue }')
table.table.table-hover
thead
tr
td Worker ID
td Queue size
td Actions
tr(each='{ queue_size, worker_id in status.WorkerQueueSize}')
td { worker_id }
td { queue_size }
td
.btn-group(
aria-label="Worker { worker_id} actions"
role="group"
)
a.btn.btn-info(
href='/#/operator/worker/{ worker_id }/logs'
)
| Logs
a.btn.btn-info(
href='/#/operator/worker/{ worker_id }/queue'
)
| Queue
div(if='{ operatorShowLogs && logs }')
table.table.table-hover
tr(each='{ logs }')
td
span.label.label-font-size(class='{ levels[Level].class }')
| { levels[Level].label }
td(style='white-space: pre')
| { Time }
td(style='white-space: pre')
| { ClusterName }
td { Message }
div(if='{ operatorShowQueue && queue }')
table.table.table-hover
tr(each='{ queue }')
td(style='white-space: pre')
| { EventTime }
td(style='white-space: pre')
| { EventType }
script.
this.levels = {
0: { class: '', label: 'Panic' },
1: { class: '', label: 'Fatal'},
2: { class: 'label-danger', label: 'Error'},
3: { class: 'label-warning', label: 'Warning'},
4: { class: 'label-primary', label: 'Info'},
5: { class: 'label-info', label: 'Debug'},
}
this.status = {
'Clusters': 0,
}
this.pollStatus = () => {
jQuery.get(
'/operator/status',
).done(data => {
this.update({status: data})
})
}
this.logs = []
this.queue = []
this.on('mount', () => {
route.exec()
})
this.pollLogs = id => {
jQuery.get(
'/operator/workers/' + id + '/logs',
).done(data => {
data.reverse()
this.update({logs: data})
})
}
this.pollQueue = id => {
jQuery.get(
'/operator/workers/' + id + '/queue',
).done(data =>
this.update({queue: data.List})
)
}
var subRoute = route.create()
subRoute('/operator/worker/*/logs', id => {
this.worker_id = id
this.operatorShowLogs = true
this.operatorShowQueue = false
this.pollLogs(this.worker_id)
})
subRoute('/operator/worker/*/queue', id => {
this.worker_id = id
this.operatorShowLogs = false
this.operatorShowQueue = true
this.pollQueue(this.worker_id)
})
subRoute('/operator', () => {
this.operatorShowLogs = false
this.operatorShowQueue = false
this.pollStatus()
})

118
ui/app/webpack.config.js Normal file
View File

@ -0,0 +1,118 @@
const DEBUG = process.env.NODE_ENV !== 'production'
const entry = ['./src/app.js']
const path = require('path')
const pkg = require('./package.json')
const webpack = require('webpack')
module.exports = {
context: path.join(__dirname, './'),
devtool: DEBUG ? 'inline-source-map' : false,
entry: entry,
mode: DEBUG ? 'development' : 'production',
target: 'web',
node: {
fs: 'empty'
},
externals: {
'$': '$',
'jquery': 'jQuery',
},
output: {
// filename: DEBUG ? 'app.js' : 'app-[hash].js'
filename: 'app.js',
library: 'App',
path: path.resolve(pkg.config.buildDir),
publicPath: DEBUG ? '/' : './',
},
plugins: [
new webpack.optimize.OccurrenceOrderPlugin(),
new webpack.HotModuleReplacementPlugin(),
new webpack.LoaderOptionsPlugin({
debug: DEBUG,
}),
// Print on rebuild when watching; see
// https://github.com/webpack/webpack/issues/1499#issuecomment-155064216
function () {
this.plugin('watch-run', (watching, callback) => {
console.log('Begin compile at ' + new Date())
callback()
})
},
],
module: {
rules: [
{
test: /\.tag\.pug$/,
loader: 'riot-tag-loader',
exclude: /node_modules/,
query: {
hot: true,
template: 'pug',
type: 'es6',
},
},
{
test: /\.tag$/,
loader: 'riot-tag-loader',
exclude: /node_modules/,
query: {
hot: false,
type: 'es6',
},
},
{
test: /\.js$/,
loader: 'babel-loader',
exclude: /node_modules/,
query: {
plugins: ['@babel/transform-runtime'],
presets: ['@babel/preset-env'],
},
},
{
test: /\.html$/,
loader: 'file-loader?name=[path][name].[ext]',
exclude: /node_modules/,
},
{
test: /\.jpe?g$|\.svg$|\.png$/,
loader: 'file-loader?name=[path][name].[ext]',
exclude: /node_modules/,
},
{
test: /\.json$/,
loader: 'json',
exclude: /node_modules/,
},
{
test: /\.(otf|eot|svg|ttf|woff|woff2)(\?v=\d+\.\d+\.\d+)?$/,
loader: 'url?limit=8192&mimetype=application/font-woff',
},
{
test: /\.json$/,
loader: 'json',
include: path.join(__dirname, 'node_modules', 'pixi.js'),
},
],
},
}

View File

@ -0,0 +1,68 @@
apiVersion: "apps/v1"
kind: "Deployment"
metadata:
name: "postgres-operator-ui"
namespace: "default"
labels:
application: "postgres-operator-ui"
team: "acid"
spec:
replicas: 1
selector:
matchLabels:
application: "postgres-operator-ui"
template:
metadata:
labels:
application: "postgres-operator-ui"
team: "acid"
spec:
serviceAccountName: postgres-operator-ui
containers:
- name: "service"
image: registry.opensource.zalan.do/acid/postgres-operator-ui:v1.2.0
ports:
- containerPort: 8081
protocol: "TCP"
readinessProbe:
httpGet:
path: "/health"
port: 8081
initialDelaySeconds: 5
timeoutSeconds: 1
resources:
limits:
cpu: "300m"
memory: "3000Mi"
requests:
cpu: "100m"
memory: "100Mi"
env:
- name: "APP_URL"
value: "http://localhost:8081"
- name: "OPERATOR_API_URL"
value: "http://localhost:8080"
- name: "TARGET_NAMESPACE"
value: "default"
- name: "TEAMS"
value: |-
[
"acid"
]
- name: "OPERATOR_UI_CONFIG"
value: |-
{
"docs_link":"https://postgres-operator.readthedocs.io/en/latest/",
"dns_format_string": "{1}-{0}.{2}",
"databases_visible": true,
"master_load_balancer_visible": true,
"nat_gateways_visible": false,
"replica_load_balancer_visible": true,
"resources_visible": true,
"users_visible": true,
"postgresql_versions": [
"11",
"10",
"9.6"
]
}

15
ui/manifests/ingress.yaml Normal file
View File

@ -0,0 +1,15 @@
apiVersion: "networking.k8s.io/v1beta1"
kind: "Ingress"
metadata:
name: "postgres-operator-ui"
namespace: "default"
labels:
application: "postgres-operator-ui"
spec:
rules:
- host: "ui.example.org"
http:
paths:
- backend:
serviceName: "postgres-operator-ui"
servicePort: 80

15
ui/manifests/service.yaml Normal file
View File

@ -0,0 +1,15 @@
apiVersion: "v1"
kind: "Service"
metadata:
name: "postgres-operator-ui"
namespace: "default"
labels:
application: "postgres-operator-ui"
spec:
type: "ClusterIP"
selector:
application: "postgres-operator-ui"
ports:
- port: 80
protocol: "TCP"
targetPort: 8081

View File

@ -0,0 +1,67 @@
apiVersion: v1
kind: ServiceAccount
metadata:
name: postgres-operator-ui
namespace: default
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
name: postgres-operator-ui
rules:
- apiGroups:
- acid.zalan.do
resources:
- postgresqls
verbs:
- create
- delete
- get
- list
- patch
- update
- apiGroups:
- ""
resources:
- pods
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- services
verbs:
- get
- list
- apiGroups:
- apps
resources:
- statefulsets
verbs:
- get
- list
- apiGroups:
- ""
resources:
- namespaces
verbs:
- get
- list
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: postgres-operator-ui
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: postgres-operator-ui
subjects:
- kind: ServiceAccount
# note: the cluster role binding needs to be defined
# for every namespace the operator-ui service account lives in.
name: postgres-operator-ui
namespace: default

View File

@ -0,0 +1,2 @@
# This version is replaced during release process.
__version__ = '2017.0.dev1'

View File

@ -0,0 +1,3 @@
from .main import main
main()

48
ui/operator_ui/backoff.py Normal file
View File

@ -0,0 +1,48 @@
import random
def expo(n: int, base=2, factor=1, max_value=None):
"""Exponential decay.
Adapted from https://github.com/litl/backoff/blob/master/backoff.py (MIT License)
Args:
base: The mathematical base of the exponentiation operation
factor: Factor to multiply the exponentation by.
max_value: The maximum value to yield. Once the value in the
true exponential sequence exceeds this, the value
of max_value will forever after be yielded.
"""
a = factor * base ** n
if max_value is None or a < max_value:
return a
else:
return max_value
def random_jitter(value, jitter=1):
"""Jitter the value a random number of milliseconds.
Copied from https://github.com/litl/backoff/blob/master/backoff.py (MIT License)
This adds up to 1 second of additional time to the original value.
Prior to backoff version 1.2 this was the default jitter behavior.
Args:
value: The unadulterated backoff value.
"""
return value + random.uniform(0, jitter)
def full_jitter(value):
"""Jitter the value across the full range (0 to value).
Copied from https://github.com/litl/backoff/blob/master/backoff.py (MIT License)
This corresponds to the "Full Jitter" algorithm specified in the
AWS blog's post on the performance of various jitter algorithms.
(http://www.awsarchitectureblog.com/2015/03/backoff.html)
Args:
value: The unadulterated backoff value.
"""
return random.uniform(0, value)

View File

@ -0,0 +1,125 @@
import logging
import re
from pathlib import Path
import kubernetes.client
import kubernetes.config
import tokens
from requests.auth import AuthBase
# default URL points to kubectl proxy
DEFAULT_CLUSTERS = 'http://localhost:8001/'
CLUSTER_ID_INVALID_CHARS = re.compile('[^a-z0-9:-]')
logger = logging.getLogger(__name__)
tokens.configure(from_file_only=True)
def generate_cluster_id(url: str):
'''Generate some "cluster ID" from given API server URL'''
for prefix in ('https://', 'http://'):
if url.startswith(prefix):
url = url[len(prefix):]
return CLUSTER_ID_INVALID_CHARS.sub('-', url.lower()).strip('-')
class StaticAuthorizationHeaderAuth(AuthBase):
'''Static authentication with given "Authorization" header'''
def __init__(self, authorization):
self.authorization = authorization
def __call__(self, request):
request.headers['Authorization'] = self.authorization
return request
class OAuthTokenAuth(AuthBase):
'''Dynamic authentication using the "tokens" library to load OAuth tokens from file
(potentially mounted from a Kubernetes secret)'''
def __init__(self, token_name):
self.token_name = token_name
tokens.manage(token_name)
def __call__(self, request):
token = tokens.get(self.token_name)
request.headers['Authorization'] = 'Bearer {}'.format(token)
return request
class Cluster:
def __init__(self, id, api_server_url, ssl_ca_cert=None, auth=None, cert_file=None, key_file=None):
self.id = id
self.api_server_url = api_server_url
self.ssl_ca_cert = ssl_ca_cert
self.auth = auth
self.cert_file = cert_file
self.key_file = key_file
class StaticClusterDiscoverer:
def __init__(self, api_server_urls: list):
self._clusters = []
if not api_server_urls:
try:
kubernetes.config.load_incluster_config()
except kubernetes.config.ConfigException:
# we are not running inside a cluster
# => assume default kubectl proxy URL
cluster = Cluster(generate_cluster_id(DEFAULT_CLUSTERS), DEFAULT_CLUSTERS)
else:
logger.info("in cluster configuration failed")
config = kubernetes.client.configuration
cluster = Cluster(
generate_cluster_id(config.host),
config.host,
ssl_ca_cert=config.ssl_ca_cert,
auth=StaticAuthorizationHeaderAuth(config.api_key['authorization']))
self._clusters.append(cluster)
else:
for api_server_url in api_server_urls:
logger.info("api server url: {}".format(api_server_url))
if 'localhost' not in api_server_url:
# TODO: hacky way of detecting whether we need a token or not
auth = OAuthTokenAuth('read-only')
else:
auth = None
self._clusters.append(Cluster(generate_cluster_id(api_server_url), api_server_url, auth=auth))
def get_clusters(self):
return self._clusters
class KubeconfigDiscoverer:
def __init__(self, kubeconfig_path: Path, contexts: set):
self._path = kubeconfig_path
self._contexts = contexts
def get_clusters(self):
# Kubernetes Python client expects "vintage" string path
config_file = str(self._path)
contexts, current_context = kubernetes.config.list_kube_config_contexts(config_file)
for context in contexts:
if self._contexts and context['name'] not in self._contexts:
# filter out
continue
config = kubernetes.client.ConfigurationObject()
kubernetes.config.load_kube_config(config_file, context=context['name'], client_configuration=config)
authorization = config.api_key.get('authorization')
if authorization:
auth = StaticAuthorizationHeaderAuth(authorization)
else:
auth = None
cluster = Cluster(
context['name'],
config.host,
ssl_ca_cert=config.ssl_ca_cert,
cert_file=config.cert_file,
key_file=config.key_file,
auth=auth)
yield cluster

1041
ui/operator_ui/main.py Normal file

File diff suppressed because it is too large Load Diff

12
ui/operator_ui/mock.py Normal file
View File

@ -0,0 +1,12 @@
import time
import json
import request
class MockCluster:
def get_pods(self):
return [{"name": "cluster-1-XFF", "role": "master", "ip": "localhost", "port": "8080"},
{"name": "cluster-1-XFE", "role": "replica", "ip": "localhost", "port": "8080"},
{"name": "cluster-1-XFS", "role": "replica", "ip": "localhost", "port": "8080"},
{"name": "cluster-2-SJE", "role": "master", "ip": "localhost", "port": "8080"}]

32
ui/operator_ui/oauth.py Normal file
View File

@ -0,0 +1,32 @@
import os
from flask_oauthlib.client import OAuthRemoteApp
CREDENTIALS_DIR = os.getenv('CREDENTIALS_DIR', '')
class OAuthRemoteAppWithRefresh(OAuthRemoteApp):
'''Same as flask_oauthlib.client.OAuthRemoteApp, but always loads client credentials from file.'''
def __init__(self, oauth, name, **kwargs):
# constructor expects some values, so make it happy..
kwargs['consumer_key'] = 'not-needed-here'
kwargs['consumer_secret'] = 'not-needed-here'
OAuthRemoteApp.__init__(self, oauth, name, **kwargs)
def refresh_credentials(self):
with open(os.path.join(CREDENTIALS_DIR, 'authcode-client-id')) as fd:
self._consumer_key = fd.read().strip()
with open(os.path.join(CREDENTIALS_DIR, 'authcode-client-secret')) as fd:
self._consumer_secret = fd.read().strip()
@property
def consumer_key(self):
self.refresh_credentials()
return self._consumer_key
@property
def consumer_secrect(self):
self.refresh_credentials()
return self._consumer_secret

View File

@ -0,0 +1,312 @@
from boto3 import client
from datetime import datetime, timezone
from furl import furl
from json import dumps
from logging import getLogger
from os import environ
from requests import Session
from urllib.parse import urljoin
from uuid import UUID
from wal_e.cmd import configure_backup_cxt
from .utils import Attrs, defaulting, these
logger = getLogger(__name__)
session = Session()
def request(cluster, path, **kwargs):
if 'timeout' not in kwargs:
# sane default timeout
kwargs['timeout'] = (5, 15)
if cluster.cert_file and cluster.key_file:
kwargs['cert'] = (cluster.cert_file, cluster.key_file)
return session.get(
urljoin(cluster.api_server_url, path),
auth=cluster.auth,
verify=cluster.ssl_ca_cert,
**kwargs
)
def request_post(cluster, path, data, **kwargs):
if 'timeout' not in kwargs:
# sane default timeout
kwargs['timeout'] = 5
if cluster.cert_file and cluster.key_file:
kwargs['cert'] = (cluster.cert_file, cluster.key_file)
return session.post(
urljoin(cluster.api_server_url, path),
data=data,
auth=cluster.auth,
verify=cluster.ssl_ca_cert,
**kwargs
)
def request_put(cluster, path, data, **kwargs):
if 'timeout' not in kwargs:
# sane default timeout
kwargs['timeout'] = 5
if cluster.cert_file and cluster.key_file:
kwargs['cert'] = (cluster.cert_file, cluster.key_file)
return session.put(
urljoin(cluster.api_server_url, path),
data=data,
auth=cluster.auth,
verify=cluster.ssl_ca_cert,
**kwargs
)
def request_delete(cluster, path, **kwargs):
if 'timeout' not in kwargs:
# sane default timeout
kwargs['timeout'] = 5
if cluster.cert_file and cluster.key_file:
kwargs['cert'] = (cluster.cert_file, cluster.key_file)
return session.delete(
urljoin(cluster.api_server_url, path),
auth=cluster.auth,
verify=cluster.ssl_ca_cert,
**kwargs
)
def resource_api_version(resource_type):
return {
'postgresqls': 'apis/acid.zalan.do/v1',
'statefulsets': 'apis/apps/v1beta1',
}.get(resource_type, 'api/v1')
def encode_labels(label_selector):
return ','.join([
f'{label}={value}'
for label, value in label_selector.items()
])
def kubernetes_url(
resource_type,
namespace='default',
resource_name=None,
label_selector=None,
):
return furl('/').add(
path=(
resource_api_version(resource_type).split('/')
+ (
['namespaces', namespace]
if namespace
else []
)
+ [resource_type]
+ (
[resource_name]
if resource_name
else []
)
),
args=(
{'labelSelector': encode_labels(label_selector)}
if label_selector
else {}
),
).url
def kubernetes_get(cluster, **kwargs):
response = request(cluster, kubernetes_url(**kwargs))
if response.status_code == 404:
return None
if response.status_code >= 400:
response.raise_for_status()
return response.json()
def read_pods(cluster, namespace, spilo_cluster):
return kubernetes_get(
cluster=cluster,
resource_type='pods',
namespace=namespace,
label_selector={'version': spilo_cluster},
)
def read_pod(cluster, namespace, resource_name):
return kubernetes_get(
cluster=cluster,
resource_type='pods',
namespace=namespace,
resource_name=resource_name,
label_selector={'application': 'spilo'},
)
def read_service(cluster, namespace, resource_name):
return kubernetes_get(
cluster=cluster,
resource_type='services',
namespace=namespace,
resource_name=resource_name,
label_selector={'application': 'spilo'},
)
def read_statefulset(cluster, namespace, resource_name):
return kubernetes_get(
cluster=cluster,
resource_type='statefulsets',
namespace=namespace,
resource_name=resource_name,
label_selector={'application': 'spilo'},
)
def read_postgresql(cluster, namespace, resource_name):
return kubernetes_get(
cluster=cluster,
resource_type='postgresqls',
namespace=namespace,
resource_name=resource_name,
)
def read_postgresqls(cluster, namespace):
return kubernetes_get(
cluster=cluster,
resource_type='postgresqls',
namespace=namespace,
)
def read_namespaces(cluster):
return kubernetes_get(
cluster=cluster,
resource_type='namespaces',
namespace=None,
)
def create_postgresql(cluster, namespace, definition):
path = kubernetes_url(
resource_type='postgresqls',
namespace=namespace,
)
try:
r = request_post(cluster, path, dumps(definition))
r.raise_for_status()
return True
except Exception as ex:
logger.exception("K8S create request failed")
return False
def apply_postgresql(cluster, namespace, resource_name, definition):
path = kubernetes_url(
resource_type='postgresqls',
namespace=namespace,
resource_name=resource_name,
)
try:
r = request_put(cluster, path, dumps(definition))
r.raise_for_status()
return True
except Exception as ex:
logger.exception("K8S create request failed")
return False
def remove_postgresql(cluster, namespace, resource_name):
path = kubernetes_url(
resource_type='postgresqls',
namespace=namespace,
resource_name=resource_name,
)
try:
r = request_delete(cluster, path)
r.raise_for_status()
return True
except Exception as ex:
logger.exception("K8S delete request failed")
return False
def read_stored_clusters(bucket, prefix, delimiter='/'):
return [
prefix['Prefix'].split('/')[-2]
for prefix in these(
client('s3').list_objects(
Bucket=bucket,
Delimiter=delimiter,
Prefix=prefix,
),
'CommonPrefixes',
)
]
def read_versions(
pg_cluster,
bucket,
s3_endpoint,
prefix,
delimiter='/',
use_aws_instance_profile=False,
):
return [
'base' if uid == 'wal' else uid
for prefix in these(
client('s3').list_objects(
Bucket=bucket,
Delimiter=delimiter,
Prefix=prefix + pg_cluster + delimiter,
),
'CommonPrefixes',
)
for uid in [prefix['Prefix'].split('/')[-2]]
if uid == 'wal' or defaulting(lambda: UUID(uid))
]
def read_basebackups(
pg_cluster,
uid,
bucket,
s3_endpoint,
prefix,
delimiter='/',
use_aws_instance_profile=False,
):
environ['WALE_S3_ENDPOINT'] = s3_endpoint
suffix = '' if uid == 'base' else '/' + uid
return [
{
key: value
for key, value in basebackup.__dict__.items()
if isinstance(value, str) or isinstance(value, int)
}
for basebackup in Attrs.call(
f=configure_backup_cxt,
aws_instance_profile=use_aws_instance_profile,
s3_prefix=f's3://{bucket}/{prefix}{pg_cluster}{suffix}/wal/',
)._backup_list(detail=True)
]
def parse_time(s: str):
return (
datetime.strptime(s, '%Y-%m-%dT%H:%M:%SZ')
.replace(tzinfo=timezone.utc)
.timestamp()
)

Binary file not shown.

After

Width:  |  Height:  |  Size: 20 KiB

View File

@ -0,0 +1,139 @@
/* http://prismjs.com/download.html?themes=prism&languages=yaml */
/**
* prism.js default theme for JavaScript, CSS and HTML
* Based on dabblet (http://dabblet.com)
* @author Lea Verou
*/
code[class*="language-"],
pre[class*="language-"] {
color: black;
background: none;
text-shadow: 0 1px white;
font-family: Consolas, Monaco, 'Andale Mono', 'Ubuntu Mono', monospace;
text-align: left;
white-space: pre;
word-spacing: normal;
word-break: normal;
word-wrap: normal;
line-height: 1.5;
-moz-tab-size: 4;
-o-tab-size: 4;
tab-size: 4;
-webkit-hyphens: none;
-moz-hyphens: none;
-ms-hyphens: none;
hyphens: none;
}
pre[class*="language-"]::-moz-selection, pre[class*="language-"] ::-moz-selection,
code[class*="language-"]::-moz-selection, code[class*="language-"] ::-moz-selection {
text-shadow: none;
background: #b3d4fc;
}
pre[class*="language-"]::selection, pre[class*="language-"] ::selection,
code[class*="language-"]::selection, code[class*="language-"] ::selection {
text-shadow: none;
background: #b3d4fc;
}
@media print {
code[class*="language-"],
pre[class*="language-"] {
text-shadow: none;
}
}
/* Code blocks */
pre[class*="language-"] {
padding: 1em;
margin: .5em 0;
overflow: auto;
}
:not(pre) > code[class*="language-"],
pre[class*="language-"] {
background: #f5f2f0;
}
/* Inline code */
:not(pre) > code[class*="language-"] {
padding: .1em;
border-radius: .3em;
white-space: normal;
}
.token.comment,
.token.prolog,
.token.doctype,
.token.cdata {
color: slategray;
}
.token.punctuation {
color: #999;
}
.namespace {
opacity: .7;
}
.token.property,
.token.tag,
.token.boolean,
.token.number,
.token.constant,
.token.symbol,
.token.deleted {
color: #905;
}
.token.selector,
.token.attr-name,
.token.string,
.token.char,
.token.builtin,
.token.inserted {
color: #690;
}
.token.operator,
.token.entity,
.token.url,
.language-css .token.string,
.style .token.string {
color: #a67f59;
background: hsla(0, 0%, 100%, .5);
}
.token.atrule,
.token.attr-value,
.token.keyword {
color: #07a;
}
.token.function {
color: #DD4A68;
}
.token.regex,
.token.important,
.token.variable {
color: #e90;
}
.token.important,
.token.bold {
font-weight: bold;
}
.token.italic {
font-style: italic;
}
.token.entity {
cursor: help;
}

File diff suppressed because one or more lines are too long

View File

@ -0,0 +1,66 @@
body {
padding-top: 70px;
}
h1, h2, h3 {
font-family: 'Open Sans', sans-serif;
}
.font-robot {
font-family: 'Roboto 300', sans-serif;
}
input:invalid {
color: red;
font-weight: 600;
}
ul.ips { list-style-type: none; margin: 0; padding: 0; overflow-x: hidden; }
ul.ips li { margin: 0; padding: 0; }
ul.ips label { margin: 0; padding: 0; }
.panel-heading.collapsible {
cursor: pointer;
}
.timeline {
cursor: pointer;
}
.panel-heading .collapsible:after {
color: grey;
content: "\e113";
float: right;
font-family: 'Glyphicons Halflings';
transition: all 0.5s;
}
.panel-heading.collapsed .collapsible:after {
transform: rotate(-180deg);
}
:not(form):invalid,select.owner:disabled {
border: 1px solid red;
box-shadow: 0 0 10px red;
}
.page-header {
margin-top: 0px;
}
.page-header h1 {
margin-top: 0px;
}
label {
font-weight: normal;
margin-top: 0;
}
.sk-spinner-pulse {
background-color: darkblue;
}
td {
vertical-align: middle !important;
}

View File

@ -0,0 +1,187 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>PostgreSQL Operator UI</title>
<!-- fonts -->
<link
href="https://fonts.googleapis.com/css?family=Open+Sans|Roboto:300,400"
rel="stylesheet"
>
<!-- bootstrap -->
<link
crossorigin="anonymous"
href="https://cdnjs.cloudflare.com/ajax/libs/twitter-bootstrap/3.3.7/css/bootstrap.min.css"
integrity="sha256-916EbMg70RQy9LHiGkXzG8hSg9EdNy97GazNG/aiY1w="
rel="stylesheet"
>
<link
crossorigin="anonymous"
href="https://cdnjs.cloudflare.com/ajax/libs/twitter-bootstrap/3.3.7/css/bootstrap-theme.min.css"
integrity="sha256-ZT4HPpdCOt2lvDkXokHuhJfdOKSPFLzeAJik5U/Q+l4="
rel="stylesheet"
>
<!-- other -->
<link
crossorigin="anonymous"
href="https://cdnjs.cloudflare.com/ajax/libs/blaze/3.5.2/blaze.min.css"
integrity="sha256-5n+FnqayL2YJQucWyxvz4SzjhRqcKvgWE/sh/4FbPao="
rel="stylesheet"
>
<link
crossorigin="anonymous"
href="https://cdnjs.cloudflare.com/ajax/libs/spinkit/1.2.5/spinkit.min.css"
integrity="sha256-JLf+H3os8xYfw2Iaq4Nv8MG6dVn1gPNv4EhSWnYG3rc="
rel="stylesheet"
/>
<link
crossorigin="anonymous"
href="https://cdnjs.cloudflare.com/ajax/libs/jquery-confirm/3.3.2/jquery-confirm.min.css"
integrity="sha256-mAmp1v6ERknmeP2oHZG53W1L+zOdSVsM25WvmZ4U+fU="
rel="stylesheet"
>
<link
crossorigin="anonymous"
href="https://cdnjs.cloudflare.com/ajax/libs/vis/4.21.0/vis.css"
integrity="sha256-I1UoFd33KHIydu88R9owFaQWzwkiZV4hXXug5aYaM28="
rel="stylesheet"
>
<!-- self-hosted -->
<link href="/css/styles.css" rel="stylesheet">
<link href="/css/prism.css" rel="stylesheet">
<link href="/favicon.png" rel="icon" type="image/png">
</head>
<script>
String.prototype.format = function() {
var formatted = this;
for(arg in arguments) {
formatted = formatted.replace("{" + arg + "}", arguments[arg]);
}
return formatted;
};
</script>
<body>
<app></app>
<!-- jQuery -->
<script
crossorigin="anonymous"
integrity="sha256-hwg4gsxgFZhOsEEamdOYGBf13FyQuiTwlAQgxVSNgt4="
src="https://code.jquery.com/jquery-3.2.1.min.js"
>
</script>
<script
crossorigin="anonymous"
integrity="sha256-0Uz1UklrpANuwqJ7M0Z54jiOE/GZwlp2EBSC6slw6j8="
src="https://cdnjs.cloudflare.com/ajax/libs/jquery-confirm/3.3.2/jquery-confirm.min.js"
>
</script>
<!-- riotjs -->
<script
crossorigin="anonymous"
integrity="sha256-L1wbqR0vvN36EcDq7wNvFk+uWm9WrMAFgSQAheWqu1g="
src="https://cdnjs.cloudflare.com/ajax/libs/riot/3.9.1/riot.min.js"
>
</script>
<script
crossorigin="anonymous"
integrity="sha256-dQVFCNWwV4WTuFCMRXLhLIITUiPzYul+Fz35TrYSbGQ="
src="https://cdn.jsdelivr.net/npm/riot-route@3.1.3/dist/route+tag.min.js"
>
</script>
<script
crossorigin="anonymous"
integrity="sha256-EGKMPXH/+n6AHqghd+K5zE3e25X6hIIY0tG30ebuv4g="
src="https://cdn.jsdelivr.net/npm/riotgear@3.5.0/dist/rg.min.js"
></script>
<!-- other -->
<script
crossorigin="anonymous"
integrity="sha384-Tc5IQib027qvyjSMfHjOMaLkfuWVxZxUPnCJA7l2mCWNIpG9mGCD8wGNIcPD7Txa"
src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/js/bootstrap.min.js"
>
</script>
<script
crossorigin="anonymous"
integrity="sha256-Daf8GuI2eLKHJlOWLRR/zRy9Clqcj4TUSumbxYH9kGI="
src="https://cdnjs.cloudflare.com/ajax/libs/clipboard.js/1.7.1/clipboard.min.js"
>
</script>
<script
crossorigin="anonymous"
integrity="sha256-0JaDbGZRXlzkFbV8Xi8ZhH/zZ6QQM0Y3dCkYZ7JYq34="
src="https://cdnjs.cloudflare.com/ajax/libs/handlebars.js/4.0.10/handlebars.min.js"
>
</script>
<script
crossorigin="anonymous"
integrity="sha256-vBJO3VoZ+1/7RMqHp9ENExY+RwfXHhi0WhX2uIOS5c0="
src="https://cdnjs.cloudflare.com/ajax/libs/humanize-duration/3.12.1/humanize-duration.js"
>
</script>
<script
crossorigin="anonymous"
integrity="sha256-wzBMoYcU9BZfRm6cQLFii4K5tkNptkER9p93W/vyCqo="
src="https://cdnjs.cloudflare.com/ajax/libs/moment.js/2.21.0/moment-with-locales.min.js"
>
</script>
<script
crossorigin="anonymous"
integrity="sha256-ff7iz7mLH5QJA9IUC44b+sqjMi7c2aTR9YO2DSzAGZo="
src="https://cdnjs.cloudflare.com/ajax/libs/vis/4.21.0/vis.js"
>
</script>
<!-- self-hosted -->
<script src="/js/prism.js"></script>
<script src="/js/build/app.js"></script>
{% if google_analytics %}
<!-- Global site tag (gtag.js) - Google Analytics -->
<script async src="{{ gtag }}"></script>
<script>
window.dataLayer = window.dataLayer || [];
function gtag(){dataLayer.push(arguments);}
gtag('js', new Date());
gtag('config', '{{ google_analytics }}');
</script>
{% endif %}
</body>
</html>

View File

@ -0,0 +1,13 @@
<html>
<head>
<title>Storing client location ...</title>
</head>
<body>
<script language="javascript">
if (document.location.hash != null && document.location.hash != "") {
localStorage.setItem("login-url-hash", document.location.hash)
}
window.location.href = document.location.pathname + "?redirect=1"
</script>
</body>
</html>

View File

@ -0,0 +1,18 @@
<html>
<head>
<title>Restoring client location ...</title>
</head>
<body>
<script language="javascript">
// /login/authorized
hash = localStorage.getItem("login-url-hash")
if (null != hash) {
localStorage.removeItem("login-url-hash")
location.href = "/" + hash
}
else {
location.href = "/"
}
</script>
</body>
</html>

12
ui/operator_ui/update.py Normal file
View File

@ -0,0 +1,12 @@
import logging
import time
import gevent
import json_delta
import requests.exceptions
from .backoff import expo, random_jitter
from .utils import get_short_error_message
logger = logging.getLogger(__name__)

146
ui/operator_ui/utils.py Normal file
View File

@ -0,0 +1,146 @@
from requests.exceptions import ConnectionError, RequestException
class Attrs:
@classmethod
def call(cls, f, **kwargs):
return f(cls(**kwargs))
def __init__(self, **kwargs):
self.attrs = kwargs
def __getattr__(self, attr):
return self.attrs.get(attr)
def get_short_error_message(e: Exception):
'''Generate a reasonable short message why the HTTP request failed'''
if isinstance(e, RequestException) and e.response is not None:
# e.g. "401 Unauthorized"
return '{} {}'.format(e.response.status_code, e.response.reason)
elif isinstance(e, ConnectionError):
# e.g. "ConnectionError" or "ConnectTimeout"
return e.__class__.__name__
else:
return str(e)
def identity(value, *_, **__):
"""
Trivial identity function: return the value passed in its first argument.
Examples:
>>> identity(42)
42
>>> list(
... filter(
... identity,
... [None, False, True, 0, 1, list(), set(), dict()],
... ),
... )
[True, 1]
"""
return value
def const(value, *_, **__):
"""
Given a value, returns a function that simply returns that value.
Example:
>>> f = const(42)
>>> f()
42
>>> f()
42
"""
return lambda *_, **__: value
def catching(computation, catcher=identity, exception=Exception):
"""
Catch exceptions.
Call the provided computation with no arguments. If it throws an exception
of the provided exception class (or any exception, if no class is provided),
return the result of calling the catcher function with the exception as the
sole argument. If no catcher function is specified, return the exception.
Examples:
Catch a KeyError and return the exception itself:
>>> catching(lambda: {'foo': 'bar'}['meh'])
KeyError('meh',)
Catch a KeyError and return a default value:
>>> catching(
... computation=lambda: {'foo': 'bar'}['meh'],
... catcher=const('nope'),
... )
'nope'
"""
try:
return computation()
except exception as e:
return catcher(e)
def defaulting(computation, default=None, exception=Exception):
"""
Like `catching`, but just return a default value if an exception is caught.
If no default value is supplied, default to None.
Examples:
Catch a KeyError and return a default value, like the `get` method:
>>> defaulting(lambda: {'foo': 'bar'}['meh'], 'nope')
'nope'
Turn a ZeroDivisionError into None:
>>> defaulting(lambda: 1/0) == None
True
"""
return catching(
computation=computation,
catcher=const(default),
exception=exception,
)
def these(what, where=None):
"""
Combinator for yielding multiple values with property access.
Yields from the values generated by an attribute of the given object, or
the values generated by the given object itself if no attribute key is
specified.
Examples:
No attribute key specified; yields from the given object:
>>> these(['foo', 'bar'])
['foo', 'bar']
An attribute key is specified; yields from the values generated by the
specified attribute's value:
object:
>>> these({'foo': ['bar', 'baz']}, 'foo')
['bar', 'baz']
An invalid attribute key is specified; yields nothing:
>>> these({'foo': ['bar', 'baz']}, 'meh')
[]
"""
if not where:
return what
return what[where] if where in what else []

14
ui/requirements.txt Normal file
View File

@ -0,0 +1,14 @@
Flask-OAuthlib==0.9.5
Flask==1.0.2
backoff==1.5.0
boto3==1.5.14
boto==2.48.0
click==6.7
furl==1.0.1
gevent==1.2.2
jq==0.1.6
json_delta>=2.0
kubernetes==3.0.0
requests==2.20.1
stups-tokens>=1.1.19
wal_e==1.1.0

117
ui/run_local.sh Executable file
View File

@ -0,0 +1,117 @@
#!/usr/bin/env bash
set -e
# NOTE: You still need to start the frontend bits of this application separately
# as starting it here as a child process would leave leftover processes on
# script termination; it appears there is some complication that does not allow
# the shell to clean up nodejs grandchild processes correctly upon script exit.
# Static bits:
export APP_URL="${API_URL-http://localhost:8081}"
export OPERATOR_API_URL="${OPERATOR_API_URL-http://localhost:8080}"
export TARGET_NAMESPACE="${TARGET_NAMESPACE-*}"
default_operator_ui_config='{
"docs_link":"https://postgres-operator.readthedocs.io/en/latest/",
"dns_format_string": "{1}-{0}.{2}",
"databases_visible": true,
"nat_gateways_visible": false,
"resources_visible": true,
"users_visible": true,
"postgresql_versions": [
"11",
"10",
"9.6"
],
"static_network_whitelist": {
"localhost": ["172.0.0.1/32"]
}
}'
export OPERATOR_UI_CONFIG="${OPERATOR_UI_CONFIG-${default_operator_ui_config}}"
# S3 backup bucket:
export SPILO_S3_BACKUP_BUCKET="postgres-backup"
# defines teams
teams='["acid"]'
export TEAMS="${TEAMS-${teams}}"
# Kubernetes API URL (e.g. minikube):
kubernetes_api_url="https://192.168.99.100:8443"
# Enable job control:
set -m
# Clean up child processes on exit:
trap 'kill $(jobs -p)' EXIT
# PostgreSQL Operator UI application name as deployed:
operator_ui_application='postgres-operator-ui'
# Hijack the PostgreSQL Operator UI pod as a proxy for its AWS instance profile
# on the pod's localhost:1234 which allows the WAL-E code to list backups there:
kubectl exec \
"$(
kubectl get pods \
--server="${kubernetes_api_url}" \
--selector="application=${operator_ui_application}" \
--output='name' \
| head --lines=1 \
| sed 's@^[^/]*/@@'
)" \
-- \
sh -c '
apk add --no-cache socat;
pkill socat;
socat -v TCP-LISTEN:1234,reuseaddr,fork,su=nobody TCP:169.254.169.254:80
' \
&
# Forward localhost:1234 to localhost:1234 on the PostgreSQL Operator UI pod to
# get to the AWS instance metadata endpoint:
echo "Port forwarding to the PostgreSQL Operator UI's instance metadata service"
kubectl port-forward \
--server="${kubernetes_api_url}" \
"$(
kubectl get pods \
--server="${kubernetes_api_url}" \
--selector="application=${operator_ui_application}" \
--output='name' \
| head --lines=1 \
| sed 's@^[^/]*/@@'
)" \
1234 \
&
# Forward localhost:8080 to localhost:8080 on the PostgreSQL Operator pod, which
# allows access to the Operator REST API
# when using helm chart use --selector='app.kubernetes.io/name=postgres-operator'
echo 'Port forwarding to the PostgreSQL Operator REST API'
kubectl port-forward \
--server="${kubernetes_api_url}" \
"$(
kubectl get pods \
--server="${kubernetes_api_url}" \
--selector='name=postgres-operator' \
--output='name' \
| head --lines=1 \
| sed 's@^[^/]*/@@'
)" \
8080 \
&
# Start a local proxy on localhost:8001 of the target Kubernetes cluster's API:
kubectl proxy &
# Start application:
python3 \
-m operator_ui \
--clusters='localhost:8001' \
$@

78
ui/setup.py Normal file
View File

@ -0,0 +1,78 @@
import sys
from setuptools import find_packages, setup
from setuptools.command.test import test as TestCommand
from pathlib import Path
def read_version(package):
with (Path(package) / '__init__.py').open() as fd:
for line in fd:
if line.startswith('__version__ = '):
return line.split()[-1].strip().strip("'")
version = read_version('operator_ui')
class PyTest(TestCommand):
user_options = [('cov-html=', None, 'Generate junit html report')]
def initialize_options(self):
TestCommand.initialize_options(self)
self.cov = None
self.pytest_args = ['--cov', 'operator_ui', '--cov-report', 'term-missing', '-v']
self.cov_html = False
def finalize_options(self):
TestCommand.finalize_options(self)
if self.cov_html:
self.pytest_args.extend(['--cov-report', 'html'])
self.pytest_args.extend(['tests'])
def run_tests(self):
import pytest
errno = pytest.main(self.pytest_args)
sys.exit(errno)
def readme():
return open('README.rst', encoding='utf-8').read()
tests_require = [
'pytest',
'pytest-cov'
]
setup(
name='operator-ui',
packages=find_packages(),
version=version,
description='PostgreSQL Kubernetes Operator UI',
long_description=readme(),
author='team-acid@zalando.de',
url='https://github.com/postgres-operator',
keywords='PostgreSQL Kubernetes Operator UI',
license='MIT',
tests_require=tests_require,
extras_require={'tests': tests_require},
cmdclass={'test': PyTest},
test_suite='tests',
classifiers=[
'Development Status :: 3',
'Intended Audience :: Developers',
'Intended Audience :: System Administrators',
'License :: OSI Approved :: MIT',
'Operating System :: OS Independent',
'Programming Language :: Python',
'Programming Language :: Python :: 3.5',
'Topic :: System :: Clustering',
'Topic :: System :: Monitoring',
],
include_package_data=True, # needed to include JavaScript (see MANIFEST.in)
entry_points={'console_scripts': ['operator-ui = operator_ui.main:main']}
)

27
ui/tox.ini Normal file
View File

@ -0,0 +1,27 @@
[tox]
envlist=py35,flake8,eslint
[tox:travis]
3.5=py35,flake8,eslint
[testenv]
deps=pytest
commands=
pip install -r requirements.txt
python setup.py test
[testenv:flake8]
deps=flake8
commands=python setup.py flake8
[testenv:eslint]
whitelist_externals=eslint
changedir=app
commands=eslint src
[flake8]
max-line-length=160
ignore=E402
[pylama]
ignore=E402