public_git
/
postgres-operator
mirror of https://github.com/zalando/postgres-operator.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

add documentation for preparedDatabases feature + minor changes

This commit is contained in:
Felix Kunde 2020-04-24 09:50:40 +02:00
parent 8cef3b2bee
commit 7a12da3e85
3 changed files with 163 additions and 37 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

132
docs/user.md
View File

@ -94,7 +94,10 @@ created on every cluster managed by the operator.
* `teams API roles`: automatically create users for every member of the team
owning the database cluster.
In the next sections, we will cover those use cases in more details.
In the next sections, we will cover those use cases in more details. Note, that
the Postgres Operator can also create databases with pre-defined owner, reader
and writer roles which saves you the manual setup. Read more in the next
chapter.
### Manifest roles
@ -216,6 +219,129 @@ to choose superusers, group roles, [PAM configuration](https://github.com/CyberD
etc. An OAuth2 token can be passed to the Teams API via a secret. The name for
this secret is configurable with the `oauth_token_secret_name` parameter.
## Prepared databases with best practice roles setup
The `users` section in the manifests only allows for creating database roles
with global privileges. Fine-grained data access control or role membership can
not be defined and must be set up by the user in the database. But, the Postgres
Operator offers a separate section to specify `preparedDatabases` that will be
bootstrapped with pre-defined owner, reader and writer roles on a database-level
and, optionally, on a schema-level, too. `preparedDatabases` also enable users
to specify PostgreSQL extensions that shall be created in the bootstrap process.
### Default database and schema
A prepared database is already created by adding an empty `preparedDatabases`
section to the manifest. The database will then be called like the Postgres
cluster manifest (`-` are replaced with `_`) and will also contain a schema
called `data`.
```yaml
spec:
preparedDatabases: {}
```
### Default NOLOGIN roles
Given an example with a specified database and schema:
```yaml
spec:
preparedDatabases:
foo:
schemas:
bar: {}
```
Postgres Operator will create the following NOLOGIN roles:
| Role name | Member of | Admin |
| -------------- | -------------- | ------------- |
| foo_owner | | admin |
| foo_reader | | foo_owner |
| foo_writer | foo_reader | foo_owner |
| foo_bar_owner | | foo_owner |
| foo_bar_reader | | foo_bar_owner |
| foo_bar_writer | foo_bar_reader | foo_bar_owner |
The `<dbname>_owner` role is the database owner and should be used when creating
new database objects. All members of the `admin` role, e.g. teams API roles, can
become the owner with the `SET ROLE` command. [Default privileges](https://www.postgresql.org/docs/12/sql-alterdefaultprivileges.html)
are configured for the owner role so that the `<dbname>_reader` role
automatically gets read-access (SELECT) to new tables and sequences and the
`<dbname>_writer` receives write-access (INSERT, UPDATE, DELETE on tables,
USAGE and UPDATE on sequences). Both get USAGE on types and EXECUTE on
functions. The same principle applies on the schema level. Note, that the
database-level roles will have access incl. default privileges on all database
schemas, too. If you don't need the dedicated schema roles - i.e. you only use
one schema - you can disable the creation like this:
```yaml
spec:
preparedDatabases:
foo:
schemas:
bar:
defaultRoles: false
```
### Default LOGIN roles
The roles described in the previous paragraph can be granted to LOGIN roles from
the `users` section in the manifest. Optionally, the Postgres Operator can also
bootstrap default LOGIN roles for the database an each schema individually.
These roles will get the `_user` suffix and they inherit all right from their
NOLOGIN counterparts.
| Role name | Member of | Admin |
| ------------------- | -------------- | ------------- |
| foo_owner_user | foo_owner | admin |
| foo_reader_user | foo_reader | foo_owner |
| foo_writer_user | foo_writer | foo_owner |
| foo_bar_owner_user | foo_bar_owner | foo_owner |
| foo_bar_reader_user | foo_bar_reader | foo_bar_owner |
| foo_bar_writer_user | foo_bar_writer | foo_bar_owner |
These default users are enabled in the manifest with the `defaultUsers` flag:
```yaml
spec:
preparedDatabases:
foo:
defaultUsers: true
schemas:
bar:
defaultUsers: true
```
### Database extensions
Prepared databases also allow for creating Postgres extensions during bootstrap.
They will be created by the database owner in the specified schema.
```yaml
spec:
preparedDatabases:
foo:
extensions:
pg_partman: public
postgis: data
```
Some extensions require SUPERUSER rights on creation unless they are not
whitelisted by the [pgextwlist](https://github.com/dimitri/pgextwlist)
extension, that is shipped with the Spilo image. To see which extensions are
on the list check the `extwlist.extension` parameter in the postgresql.conf
file.
```bash
SHOW extwlist.extensions;
```
Make sure that `pgextlist` is also listed under `shared_preload_libraries` in
the PostgreSQL configuration. Then the database owner should be able to create
the extension specified in the manifest.
## Resource definition
The compute resources to be used for the Postgres containers in the pods can be
@ -584,8 +710,8 @@ don't know the value, use `103` which is the GID from the default spilo image
OpenShift allocates the users and groups dynamically (based on scc), and their
range is different in every namespace. Due to this dynamic behaviour, it's not
trivial to know at deploy time the uid/gid of the user in the cluster.
Therefore, instead of using a global `spilo_fsgroup` setting, use the `spiloFSGroup` field
per Postgres cluster.
Therefore, instead of using a global `spilo_fsgroup` setting, use the
`spiloFSGroup` field per Postgres cluster.
Upload the cert as a kubernetes secret:
```sh

44
pkg/cluster/database.go
View File

@ -27,7 +27,7 @@ const (
WHERE a.rolname = ANY($1)
ORDER BY 1;`
getDatabasesSQL = `SELECT datname, pg_get_userbyid(datdba) AS owner FROM pg_database;`
getDatabasesSQL = `SELECT databaseName, pg_get_userbyid(datdba) AS owner FROM pg_database;`
getSchemasSQL = `SELECT n.nspname AS dbschema FROM pg_catalog.pg_namespace n
WHERE n.nspname !~ '^pg_' AND n.nspname <> 'information_schema' ORDER BY 1`
getExtensionsSQL = `SELECT e.extname, n.nspname FROM pg_catalog.pg_extension e
@ -235,12 +235,12 @@ func (c *Cluster) getDatabases() (dbs map[string]string, err error) {
dbs = make(map[string]string)
for rows.Next() {
var datname, owner string
var databaseName, owner string
if err = rows.Scan(&datname, &owner); err != nil {
if err = rows.Scan(&databaseName, &owner); err != nil {
return nil, fmt.Errorf("error when processing row: %v", err)
}
dbs[datname] = owner
dbs[databaseName] = owner
}
return dbs, err
@ -248,37 +248,37 @@ func (c *Cluster) getDatabases() (dbs map[string]string, err error) {
// executeCreateDatabase creates new database with the given owner.
// The caller is responsible for opening and closing the database connection.
func (c *Cluster) executeCreateDatabase(datname, owner string) error {
return c.execCreateOrAlterDatabase(datname, owner, createDatabaseSQL,
func (c *Cluster) executeCreateDatabase(databaseName, owner string) error {
return c.execCreateOrAlterDatabase(databaseName, owner, createDatabaseSQL,
"creating database", "create database")
}
// executeAlterDatabaseOwner changes the owner of the given database.
// The caller is responsible for opening and closing the database connection.
func (c *Cluster) executeAlterDatabaseOwner(datname string, owner string) error {
return c.execCreateOrAlterDatabase(datname, owner, alterDatabaseOwnerSQL,
func (c *Cluster) executeAlterDatabaseOwner(databaseName string, owner string) error {
return c.execCreateOrAlterDatabase(databaseName, owner, alterDatabaseOwnerSQL,
"changing owner for database", "alter database owner")
}
func (c *Cluster) execCreateOrAlterDatabase(datname, owner, statement, doing, operation string) error {
if !c.databaseNameOwnerValid(datname, owner) {
func (c *Cluster) execCreateOrAlterDatabase(databaseName, owner, statement, doing, operation string) error {
if !c.databaseNameOwnerValid(databaseName, owner) {
return nil
}
c.logger.Infof("%s %q owner %q", doing, datname, owner)
if _, err := c.pgDb.Exec(fmt.Sprintf(statement, datname, owner)); err != nil {
c.logger.Infof("%s %q owner %q", doing, databaseName, owner)
if _, err := c.pgDb.Exec(fmt.Sprintf(statement, databaseName, owner)); err != nil {
return fmt.Errorf("could not execute %s: %v", operation, err)
}
return nil
}
func (c *Cluster) databaseNameOwnerValid(datname, owner string) bool {
func (c *Cluster) databaseNameOwnerValid(databaseName, owner string) bool {
if _, ok := c.pgUsers[owner]; !ok {
c.logger.Infof("skipping creation of the %q database, user %q does not exist", datname, owner)
c.logger.Infof("skipping creation of the %q database, user %q does not exist", databaseName, owner)
return false
}
if !databaseNameRegexp.MatchString(datname) {
c.logger.Infof("database %q has invalid name", datname)
if !databaseNameRegexp.MatchString(databaseName) {
c.logger.Infof("database %q has invalid name", databaseName)
return false
}
return true
@ -320,12 +320,12 @@ func (c *Cluster) getSchemas() (schemas []string, err error) {
// executeCreateDatabaseSchema creates new database schema with the given owner.
// The caller is responsible for opening and closing the database connection.
func (c *Cluster) executeCreateDatabaseSchema(datname, schemaName, dbOwner string, schemaOwner string) error {
return c.execCreateDatabaseSchema(datname, schemaName, dbOwner, schemaOwner, createDatabaseSchemaSQL,
func (c *Cluster) executeCreateDatabaseSchema(databaseName, schemaName, dbOwner string, schemaOwner string) error {
return c.execCreateDatabaseSchema(databaseName, schemaName, dbOwner, schemaOwner, createDatabaseSchemaSQL,
"creating database schema", "create database schema")
}
func (c *Cluster) execCreateDatabaseSchema(datname, schemaName, dbOwner, schemaOwner, statement, doing, operation string) error {
func (c *Cluster) execCreateDatabaseSchema(databaseName, schemaName, dbOwner, schemaOwner, statement, doing, operation string) error {
if !c.databaseSchemaNameValid(schemaName) {
return nil
}
@ -335,10 +335,10 @@ func (c *Cluster) execCreateDatabaseSchema(datname, schemaName, dbOwner, schemaO
}
// set default privileges for schema
c.execAlterSchemaDefaultPrivileges(schemaName, schemaOwner, datname)
c.execAlterSchemaDefaultPrivileges(schemaName, schemaOwner, databaseName)
if schemaOwner != dbOwner {
c.execAlterSchemaDefaultPrivileges(schemaName, dbOwner, datname+"_"+schemaName)
c.execAlterSchemaDefaultPrivileges(schemaName, schemaOwner, datname+"_"+schemaName)
c.execAlterSchemaDefaultPrivileges(schemaName, dbOwner, databaseName+"_"+schemaName)
c.execAlterSchemaDefaultPrivileges(schemaName, schemaOwner, databaseName+"_"+schemaName)
}
return nil

24
pkg/cluster/sync.go
View File

@ -560,12 +560,12 @@ func (c *Cluster) syncDatabases() error {
}
}
for datname, newOwner := range c.Spec.Databases {
currentOwner, exists := currentDatabases[datname]
for databaseName, newOwner := range c.Spec.Databases {
currentOwner, exists := currentDatabases[databaseName]
if !exists {
createDatabases[datname] = newOwner
createDatabases[databaseName] = newOwner
} else if currentOwner != newOwner {
alterOwnerDatabases[datname] = newOwner
alterOwnerDatabases[databaseName] = newOwner
}
}
@ -573,13 +573,13 @@ func (c *Cluster) syncDatabases() error {
return nil
}
for datname, owner := range createDatabases {
if err = c.executeCreateDatabase(datname, owner); err != nil {
for databaseName, owner := range createDatabases {
if err = c.executeCreateDatabase(databaseName, owner); err != nil {
return err
}
}
for datname, owner := range alterOwnerDatabases {
if err = c.executeAlterDatabaseOwner(datname, owner); err != nil {
for databaseName, owner := range alterOwnerDatabases {
if err = c.executeAlterDatabaseOwner(databaseName, owner); err != nil {
return err
}
}
@ -620,7 +620,7 @@ func (c *Cluster) syncPreparedDatabases() error {
return nil
}
func (c *Cluster) syncPreparedSchemas(datname string, preparedSchemas map[string]acidv1.PreparedSchema) error {
func (c *Cluster) syncPreparedSchemas(databaseName string, preparedSchemas map[string]acidv1.PreparedSchema) error {
c.setProcessName("syncing prepared schemas")
currentSchemas, err := c.getSchemas()
@ -637,13 +637,13 @@ func (c *Cluster) syncPreparedSchemas(datname string, preparedSchemas map[string
if createPreparedSchemas, equal := util.SubstractStringSlices(schemas, currentSchemas); !equal {
for _, schemaName := range createPreparedSchemas {
owner := constants.OwnerRoleNameSuffix
dbOwner := datname + owner
dbOwner := databaseName + owner
if preparedSchemas[schemaName].DefaultRoles == nil || *preparedSchemas[schemaName].DefaultRoles {
owner = datname + "_" + schemaName + owner
owner = databaseName + "_" + schemaName + owner
} else {
owner = dbOwner
}
if err = c.executeCreateDatabaseSchema(datname, schemaName, dbOwner, owner); err != nil {
if err = c.executeCreateDatabaseSchema(databaseName, schemaName, dbOwner, owner); err != nil {
return err
}
}