public_git
/
democratic-csi
mirror of https://github.com/democratic-csi/democratic-csi.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Merge branch 'next' of github.com:democratic-csi/democratic-csi into next

This commit is contained in:
Travis Glenn Hansen 2022-04-20 11:42:50 -06:00
parent 7f579d9a7c 167e610e77
commit c55f3957ac
5 changed files with 337 additions and 53 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
README.md
View File

@ -215,6 +215,9 @@ Ensure the following services are configurged and running:
- `curl --header "Accept: application/json" --user root:<password> 'http(s)://<ip>/api/v2.0/iscsi/portal'`
- `curl --header "Accept: application/json" --user root:<password> 'http(s)://<ip>/api/v2.0/iscsi/initiator'`
- `curl --header "Accept: application/json" --user root:<password> 'http(s)://<ip>/api/v2.0/iscsi/auth'`
- The maximum number of volumes is limited to 255 by default on FreeBSD (physical devices such as disks and CD-ROM drives count against this value).
Be sure to properly adjust both [tunables](https://www.freebsd.org/cgi/man.cgi?query=ctl&sektion=4#end) `kern.cam.ctl.max_ports` and `kern.cam.ctl.max_luns` to avoid running out of resources when dynamically provisioning iSCSI volumes on FreeNAS or TrueNAS Core.
- smb
If you would prefer you can configure `democratic-csi` to use a
@ -301,7 +304,7 @@ smbpasswd -L -a smbroot
### Synology (synology-iscsi)
Ensure iscsi manager has been installed and is generally setup/configured.
Ensure iscsi manager has been installed and is generally setup/configured. DSM 6.3+ is supported.
## Helm Installation

138
docs/storage-class-parameters.md Normal file
View File

@ -0,0 +1,138 @@
# Storage Class Parameters
Some drivers support different settings for volumes. These can be configured via the driver configuration and/or storage
classes.
## `synology-iscsi`
The `synology-iscsi` driver supports several storage class parameters. Note however that not all parameters/values are
supported for all backing file systems and LUN type. The following options are available:
### Configure Storage Classes
```yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: synology-iscsi
parameters:
fsType: ext4
# The following options affect the LUN representing the volume. These options are passed directly to the Synology API.
# The following options are known.
lunTemplate: |
type: BLUN # Btrfs thin provisioning
type: BLUN_THICK # Btrfs thick provisioning
type: THIN # Ext4 thin provisioning
type: ADV # Ext4 thin provisioning with legacy advanced feature set
type: FILE # Ext4 thick provisioning
description: Some Description
# Only for thick provisioned volumes. Known values:
# 0: Buffered Writes
# 3: Direct Write
direct_io_pattern: 0
# Device Attributes. See below for more info
dev_attribs:
- dev_attrib: emulate_tpws
enable: 1
- ...
# The following options affect the iSCSI target. These options will be passed directly to the Synology API.
# The following options are known.
targetTemplate: |
has_header_checksum: false
has_data_checksum: false
# Note that this option requires a compatible filesystem. Use 0 for unlimited sessions.
max_sessions: 0
multi_sessions: true
max_recv_seg_bytes: 262144
max_send_seg_bytes: 262144
# Use this to disable authentication. To configure authentication see below
auth_type: 0
```
#### About LUN Types
The availability of the different types of LUNs depends on the filesystem used on your Synology volume. For Btrfs volumes
you can use `BLUN` and `BLUN_THICK` volumes. For Ext4 volumes you can use `THIN`, `ADV` or `FILE` volumes. These
correspond to the options available in the UI.
#### About `dev_attribs`
Most of the LUN options are configured via the `dev_attribs` list. This list can be specified both in the `lunTemplate`
of the global configuration and in the `lunTemplate` of the `StorageClass`. If both lists are present they will be merged
(with the `StorageClass` taking precedence). The following `dev_attribs` are known to work:
- `emulate_tpws`: Hardware-assisted zeroing
- `emulate_caw`: Hardware-assisted locking
- `emulate_3pc`: Hardware-assisted data transfer
- `emulate_tpu`: Space Reclamation
- `emulate_fua_write`: Enable the FUA iSCSI command (DSM 7+)
- `emulate_sync_cache`: Enable the Sync Cache iSCSI command (DSM 7+)
- `can_snapshot`: Enable snapshots for this volume. Only works for thin provisioned volumes.
### Configure Snapshot Classes
`synology-iscsi` can also configure different parameters on snapshot classes:
```yaml
apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshotClass
metadata:
name: synology-iscsi-snapshot
parameters:
# This inline yaml object will be passed to the Synology API when creating the snapshot.
lunSnapshotTemplate: |
is_locked: true
# https://kb.synology.com/en-me/DSM/tutorial/What_is_file_system_consistent_snapshot
# Note that app consistent snapshots require a working Synology Storage Console. Otherwise both values will have
# equivalent behavior.
is_app_consistent: true
...
```
Note that it is currently not supported by Synology devices to restore a snapshot onto a different volume. You can
create volumes from snapshots, but you should use the same `StorageClass` as the original volume of the snapshot did.
### Enabling CHAP Authentication
You can enable CHAP Authentication for `StorageClass`es by supplying an appropriate `StorageClass` secret (see the
[documentation](https://kubernetes-csi.github.io/docs/secrets-and-credentials-storage-class.html) for more details). You
can use the same password for alle volumes of a `StorageClass` or use different passwords per volume.
```yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: synology-iscsi-chap
parameters:
fsType: ext4
lunTemplate: |
type: BLUN
description: iSCSI volumes with CHAP Authentication
secrets:
# Use this to configure a single set of credentials for all volumes of this StorageClass
csi.storage.k8s.io/provisioner-secret-name: chap-secret
csi.storage.k8s.io/provisioner-secret-namespace: default
# Use substitutions to use different credentials for volumes based on the PVC
csi.storage.k8s.io/provisioner-secret-name: "${pvc.name}-chap-secret"
csi.storage.k8s.io/provisioner-secret-namespace: "${pvc.namespace}"
...
---
# Use a secret like this to supply CHAP credentials.
apiVersion: v1
kind: Secret
metadata:
name: chap-secret
stringData:
# Client Credentials
user: client
password: MySecretPassword
# Mutual CHAP Credentials. If these are specified mutual CHAP will be enabled.
mutualUser: server
mutualPassword: MyOtherPassword
```
Note that CHAP authentication will only be enabled if the secret contains a username and password. If e.g. a password is
missing CHAP authentication will not be enabled (but the volume will still be created). You cannot automatically
enable/disable CHAP or change the password after the volume has been created.
If the secret itself is referenced but not present, the volume will not be created.

8
examples/synology-iscsi.yaml
View File

@ -10,9 +10,9 @@ httpConnection:
session: "democratic-csi"
serialize: true
synology:
# choose the proper volume for your system
volume: /volume1
# Choose the DSM volume this driver operates on. The default value is /volume1.
# synology:
# volume: /volume1
iscsi:
targetPortal: "server[:port]"
@ -31,6 +31,8 @@ iscsi:
# documented below are several blocks
# pick the option appropriate for you based on what your backing fs is and desired features
# you do not need to alter dev_attribs under normal circumstances but they may be altered in advanced use-cases
# These options can also be configured per storage-class:
# See https://github.com/democratic-csi/democratic-csi/blob/master/docs/storage-class-parameters.md
lunTemplate:
# btrfs thin provisioning
type: "BLUN"

77
src/driver/controller-synology/http/index.js
View File

@ -4,10 +4,40 @@ const https = require("https");
const { axios_request, stringify } = require("../../../utils/general");
const Mutex = require("async-mutex").Mutex;
const registry = require("../../../utils/registry");
const { GrpcError, grpc } = require("../../../utils/grpc");
const USER_AGENT = "democratic-csi";
const __REGISTRY_NS__ = "SynologyHttpClient";
SYNO_ERRORS = {
18990002: { status: grpc.status.RESOURCE_EXHAUSTED, message: "The synology volume is out of disk space." },
18990318: { status: grpc.status.INVALID_ARGUMENT, message: "The requested lun type is incompatible with the Synology filesystem." },
18990538: { status: grpc.status.ALREADY_EXISTS, message: "A LUN with this name already exists." },
18990541: { status: grpc.status.RESOURCE_EXHAUSTED, message: "The maximum number of LUNS has been reached." },
18990542: { status: grpc.status.RESOURCE_EXHAUSTED, message: "The maximum number if iSCSI target has been reached." },
18990744: { status: grpc.status.ALREADY_EXISTS, message: "An iSCSI target with this name already exists." },
18990532: { status: grpc.status.NOT_FOUND, message: "No such snapshot." },
18990500: { status: grpc.status.INVALID_ARGUMENT, message: "Bad LUN type" },
18990543: { status: grpc.status.RESOURCE_EXHAUSTED, message: "Maximum number of snapshots reached." },
18990635: { status: grpc.status.INVALID_ARGUMENT, message: "Invalid ioPolicy." }
}
class SynologyError extends GrpcError {
constructor(code, httpCode = undefined) {
super(0, "");
this.synoCode = code;
this.httpCode = httpCode;
if (code > 0) {
const error = SYNO_ERRORS[code]
this.code = error?.status ?? grpc.status.UNKNOWN;
this.message = error?.message ?? `An unknown error occurred when executing a synology command (code = ${code}).`;
} else {
this.code = grpc.status.UNKNOWN;
this.message = `The synology webserver returned a status code ${httpCode}`;
}
}
}
class SynologyHttpClient {
constructor(options = {}) {
this.options = JSON.parse(JSON.stringify(options));
@ -149,7 +179,7 @@ class SynologyHttpClient {
}
if (response.statusCode > 299 || response.statusCode < 200) {
reject(response);
reject(new SynologyError(null, response.statusCode))
}
if (response.body.success === false) {
@ -157,7 +187,7 @@ class SynologyHttpClient {
if (response.body.error.code == 119 && sid == client.sid) {
client.sid = null;
}
reject(response);
reject(new SynologyError(response.body.error.code, response.statusCode));
}
resolve(response);
@ -293,19 +323,19 @@ class SynologyHttpClient {
return snapshots;
}
async GetSnapshotByLunIDAndName(lun_id, name) {
async GetSnapshotByLunUUIDAndName(lun_uuid, name) {
const get_snapshot_info = {
lid: lun_id, //check?
api: "SYNO.Core.Storage.iSCSILUN",
method: "load_snapshot",
api: "SYNO.Core.ISCSI.LUN",
method: "list_snapshot",
version: 1,
src_lun_uuid: JSON.stringify(lun_uuid),
};
let response = await this.do_request("GET", "entry.cgi", get_snapshot_info);
if (response.body.data) {
let snapshot = response.body.data.find((i) => {
return i.desc == name;
if (response.body.data.snapshots) {
let snapshot = response.body.data.snapshots.find((i) => {
return i.description == name;
});
if (snapshot) {
@ -314,18 +344,18 @@ class SynologyHttpClient {
}
}
async GetSnapshotByLunIDAndSnapshotUUID(lun_id, snapshot_uuid) {
async GetSnapshotByLunUUIDAndSnapshotUUID(lun_uuid, snapshot_uuid) {
const get_snapshot_info = {
lid: lun_id, //check?
api: "SYNO.Core.Storage.iSCSILUN",
method: "load_snapshot",
api: "SYNO.Core.ISCSI.LUN",
method: "list_snapshot",
version: 1,
src_lun_uuid: JSON.stringify(lun_uuid),
};
let response = await this.do_request("GET", "entry.cgi", get_snapshot_info);
if (response.body.data) {
let snapshot = response.body.data.find((i) => {
if (response.body.data.snapshots) {
let snapshot = response.body.data.snapshots.find((i) => {
return i.uuid == snapshot_uuid;
});
@ -412,7 +442,7 @@ class SynologyHttpClient {
response = await this.do_request("GET", "entry.cgi", iscsi_lun_create);
return response.body.data.uuid;
} catch (err) {
if ([18990538].includes(err.body.error.code)) {
if (err.synoCode === 18990538) {
response = await this.do_request("GET", "entry.cgi", lun_list);
let lun = response.body.data.luns.find((i) => {
return i.name == iscsi_lun_create.name;
@ -503,7 +533,7 @@ class SynologyHttpClient {
return response.body.data.target_id;
} catch (err) {
if ([18990744].includes(err.body.error.code)) {
if (err.synoCode === 18990744) {
//do lookup
const iscsi_target_list = {
api: "SYNO.Core.ISCSI.Target",
@ -549,7 +579,7 @@ class SynologyHttpClient {
/**
* 18990710 = non-existant
*/
//if (![18990710].includes(err.body.error.code)) {
//if (err.synoCode !== 18990710) {
throw err;
//}
}
@ -572,20 +602,24 @@ class SynologyHttpClient {
);
}
async CreateClonedVolume(src_lun_uuid, dst_lun_name) {
async CreateClonedVolume(src_lun_uuid, dst_lun_name, dst_location, description) {
const create_cloned_volume = {
api: "SYNO.Core.ISCSI.LUN",
version: 1,
method: "clone",
src_lun_uuid: JSON.stringify(src_lun_uuid), // src lun uuid
dst_lun_name: dst_lun_name, // dst lun name
dst_location: dst_location,
is_same_pool: true, // always true? string?
clone_type: "democratic-csi", // check
};
if (description) {
create_cloned_volume.description = description;
}
return await this.do_request("GET", "entry.cgi", create_cloned_volume);
}
async CreateVolumeFromSnapshot(src_lun_uuid, snapshot_uuid, cloned_lun_name) {
async CreateVolumeFromSnapshot(src_lun_uuid, snapshot_uuid, cloned_lun_name, description) {
const create_volume_from_snapshot = {
api: "SYNO.Core.ISCSI.LUN",
version: 1,
@ -595,6 +629,9 @@ class SynologyHttpClient {
cloned_lun_name: cloned_lun_name, // cloned lun name
clone_type: "democratic-csi", // check
};
if (description) {
create_volume_from_snapshot.description = description;
}
return await this.do_request(
"GET",
"entry.cgi",

162
src/driver/controller-synology/index.js
View File

@ -3,6 +3,8 @@ const { GrpcError, grpc } = require("../../utils/grpc");
const registry = require("../../utils/registry");
const SynologyHttpClient = require("./http").SynologyHttpClient;
const semver = require("semver");
const sleep = require("../../utils/general").sleep;
const yaml = require("js-yaml");
const GeneralUtils = require("../../utils/general");
const __REGISTRY_NS__ = "ControllerSynologyDriver";
@ -142,6 +144,23 @@ class ControllerSynologyDriver extends CsiBaseDriver {
}
}
getObjectFromDevAttribs(list = []) {
if (!list) {
return {}
}
return list.reduce(
(obj, item) => Object.assign(obj, {[item.dev_attrib]: item.enable}), {}
)
}
getDevAttribsFromObject(obj, keepNull = false) {
return Object.entries(obj).filter(
e => keepNull || (e[1] != null)
).map(
e => ({dev_attrib: e[0], enable: e[1]})
);
}
buildIscsiName(name) {
let iscsiName = name;
if (this.options.iscsi.namePrefix) {
@ -155,6 +174,25 @@ class ControllerSynologyDriver extends CsiBaseDriver {
return iscsiName.toLowerCase();
}
/**
* Returns the value for the 'location' parameter indicating on which volume
* a LUN is to be created.
*
* @param {Object} parameters - Parameters received from a StorageClass
* @param {String} parameters.volume - The volume specified by the StorageClass
* @returns {String} The location of the volume.
*/
getLocation() {
let location = this.options?.synology?.volume;
if (location === undefined) {
location = "volume1"
}
if (!location.startsWith('/')) {
location = "/" + location
}
return location
}
assertCapabilities(capabilities) {
const driverResourceType = this.getDriverResourceType();
this.ctx.logger.verbose("validating capabilities: %j", capabilities);
@ -312,6 +350,7 @@ class ControllerSynologyDriver extends CsiBaseDriver {
}
let volume_context = {};
const normalizedParameters = driver.getNormalizedParameters(call.request.parameters);
switch (driver.getDriverShareType()) {
case "nfs":
// TODO: create volume here
@ -329,6 +368,7 @@ class ControllerSynologyDriver extends CsiBaseDriver {
break;
case "iscsi":
let iscsiName = driver.buildIscsiName(name);
let storageClassTemplate;
let data;
let target;
let lun_mapping;
@ -363,13 +403,12 @@ class ControllerSynologyDriver extends CsiBaseDriver {
if (volume_content_source) {
let src_lun_uuid;
let src_lun_id;
switch (volume_content_source.type) {
case "snapshot":
let parts = volume_content_source.snapshot.snapshot_id.split("/");
src_lun_id = parts[2];
if (!src_lun_id) {
src_lun_uuid = parts[2];
if (!src_lun_uuid) {
throw new GrpcError(
grpc.status.NOT_FOUND,
`invalid snapshot_id: ${volume_content_source.snapshot.snapshot_id}`
@ -384,11 +423,14 @@ class ControllerSynologyDriver extends CsiBaseDriver {
);
}
let src_lun = await httpClient.GetLunByID(src_lun_id);
src_lun_uuid = src_lun.uuid;
// This is for backwards compatibility. Previous versions of this driver used the LUN ID instead of the
// UUID. If this is the case we need to get the LUN UUID before we can proceed.
if (!src_lun_uuid.includes("-")) {
src_lun_uuid = await httpClient.GetLunByID(src_lun_uuid).uuid;
}
let snapshot = await httpClient.GetSnapshotByLunIDAndSnapshotUUID(
src_lun_id,
let snapshot = await httpClient.GetSnapshotByLunUUIDAndSnapshotUUID(
src_lun_uuid,
snapshot_uuid
);
if (!snapshot) {
@ -403,7 +445,8 @@ class ControllerSynologyDriver extends CsiBaseDriver {
await httpClient.CreateVolumeFromSnapshot(
src_lun_uuid,
snapshot_uuid,
iscsiName
iscsiName,
normalizedParameters.description
);
}
break;
@ -427,7 +470,12 @@ class ControllerSynologyDriver extends CsiBaseDriver {
`invalid volume_id: ${volume_content_source.volume.volume_id}`
);
}
await httpClient.CreateClonedVolume(src_lun_uuid, iscsiName);
await httpClient.CreateClonedVolume(
src_lun_uuid,
iscsiName,
driver.getLocation(),
normalizedParameters.description
);
}
break;
default:
@ -446,20 +494,62 @@ class ControllerSynologyDriver extends CsiBaseDriver {
}
} else {
// create lun
data = Object.assign({}, driver.options.iscsi.lunTemplate, {
name: iscsiName,
location: driver.options.synology.volume,
size: capacity_bytes,
});
lun_uuid = await httpClient.CreateLun(data);
try {
storageClassTemplate = yaml.load(normalizedParameters.lunTemplate ?? "")
const devAttribs = driver.getDevAttribsFromObject(Object.assign(
{},
driver.getObjectFromDevAttribs(driver.options.iscsi.lunTemplate?.dev_attribs),
driver.getObjectFromDevAttribs(storageClassTemplate?.dev_attribs)
))
data = Object.assign({}, driver.options.iscsi.lunTemplate, storageClassTemplate, {
name: iscsiName,
location: driver.getLocation(),
size: capacity_bytes,
dev_attribs: devAttribs
});
lun_uuid = await httpClient.CreateLun(data);
} catch (err) {
if (err instanceof yaml.YAMLException) {
throw new GrpcError(
grpc.status.INVALID_ARGUMENT,
`The lunTemplate on StorageClass is not a valid YAML document.`
);
} else {
throw err
}
}
}
// create target
let iqn = driver.options.iscsi.baseiqn + iscsiName;
data = Object.assign({}, driver.options.iscsi.targetTemplate, {
try {
storageClassTemplate = yaml.load(normalizedParameters.targetTemplate ?? "")
} catch (err) {
throw new GrpcError(
grpc.status.INVALID_ARGUMENT,
`The targetTemplate on StorageClass is not a valid YAML document.`
);
}
data = Object.assign({}, driver.options.iscsi.targetTemplate, storageClassTemplate, {
name: iscsiName,
iqn,
});
if ('user' in call.request.secrets && 'password' in call.request.secrets) {
data.user = call.request.secrets.user;
data.password = call.request.secrets.password;
data.chap = true;
if ('mutualUser' in call.request.secrets && 'mutualPassword' in call.request.secrets) {
data.mutual_user = call.request.secrets.mutualUser;
data.mutual_password = call.request.secrets.mutualPassword;
data.auth_type = 2;
data.mutual_chap = true;
} else {
data.auth_type = 1;
data.mutual_chap = false;
}
}
let target_id = await httpClient.CreateTarget(data);
//target = await httpClient.GetTargetByTargetID(target_id);
target = await httpClient.GetTargetByIQN(iqn);
@ -739,8 +829,9 @@ class ControllerSynologyDriver extends CsiBaseDriver {
async GetCapacity(call) {
const driver = this;
const httpClient = await driver.getHttpClient();
const location = driver.getLocation();
if (!driver.options.synology.volume) {
if (!location) {
throw new GrpcError(
grpc.status.FAILED_PRECONDITION,
`invalid configuration: missing volume`
@ -755,9 +846,7 @@ class ControllerSynologyDriver extends CsiBaseDriver {
}
}
let response = await httpClient.GetVolumeInfo(
driver.options.synology.volume
);
let response = await httpClient.GetVolumeInfo(location);
return { available_capacity: response.body.data.volume.size_free_byte };
}
@ -850,16 +939,26 @@ class ControllerSynologyDriver extends CsiBaseDriver {
// check for already exists
let snapshot;
snapshot = await httpClient.GetSnapshotByLunIDAndName(lun.lun_id, name);
let snapshotClassTemplate;
snapshot = await httpClient.GetSnapshotByLunUUIDAndName(lun.uuid, name);
if (!snapshot) {
let data = Object.assign({}, driver.options.iscsi.lunSnapshotTemplate, {
const normalizedParameters = driver.getNormalizedParameters(call.request.parameters);
try {
snapshotClassTemplate = yaml.load(normalizedParameters.lunSnapshotTemplate ?? "");
} catch (err) {
throw new GrpcError(
grpc.status.INVALID_ARGUMENT,
`The snapshotTemplate on VolumeSnapshotClass is not a valid YAML document.`
);
}
let data = Object.assign({}, driver.options.iscsi.lunSnapshotTemplate, snapshotClassTemplate, {
src_lun_uuid: lun.uuid,
taken_by: "democratic-csi",
description: name, //check
});
await httpClient.CreateSnapshot(data);
snapshot = await httpClient.GetSnapshotByLunIDAndName(lun.lun_id, name);
snapshot = await httpClient.GetSnapshotByLunUUIDAndName(lun.uuid, name);
if (!snapshot) {
throw new Error(`failed to create snapshot`);
@ -873,7 +972,7 @@ class ControllerSynologyDriver extends CsiBaseDriver {
* is needed to create a volume from this snapshot.
*/
size_bytes: snapshot.total_size,
snapshot_id: `/lun/${lun.lun_id}/${snapshot.uuid}`, // add shanpshot_uuid //fixme
snapshot_id: `/lun/${lun.uuid}/${snapshot.uuid}`,
source_volume_id: source_volume_id,
//https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/timestamp.proto
creation_time: {
@ -910,8 +1009,8 @@ class ControllerSynologyDriver extends CsiBaseDriver {
}
let parts = snapshot_id.split("/");
let lun_id = parts[2];
if (!lun_id) {
let lun_uuid = parts[2];
if (!lun_uuid) {
return {};
}
@ -920,9 +1019,14 @@ class ControllerSynologyDriver extends CsiBaseDriver {
return {};
}
// TODO: delete snapshot
let snapshot = await httpClient.GetSnapshotByLunIDAndSnapshotUUID(
lun_id,
// This is for backwards compatibility. Previous versions of this driver used the LUN ID instead of the UUID. If
// this is the case we need to get the LUN UUID before we can proceed.
if (!lun_uuid.includes("-")) {
lun_uuid = await httpClient.GetLunByID(lun_uuid).uuid;
}
let snapshot = await httpClient.GetSnapshotByLunUUIDAndSnapshotUUID(
lun_uuid,
snapshot_uuid
);