diff --git a/README.md b/README.md index b15a62a..90552c8 100644 --- a/README.md +++ b/README.md @@ -6,17 +6,17 @@ providing storage for various container orchestration systems (ie: Kubernetes). The current focus is providing storage via iscsi/nfs from zfs-based storage systems, predominantly `FreeNAS / TrueNAS` and `ZoL` on `Ubuntu`. -The current drivers implement depth and breadth of the `csi` spec, so you have -access to resizing, snapshots, etc, etc. +The current drivers implement the depth and breadth of the `csi` spec, so you +have access to resizing, snapshots, clones, etc functionality. `democratic-csi` is 2 things: - several implementations of `csi` drivers - - freenas-nfs (manages zfs datasets to share over nfs) - - freenas-iscsi (manages zfs zvols to share over iscsi) - - zfs-generic-nfs (works with any ZoL installation...ie: Ubuntu) - - zfs-generic-iscsi (works with any ZoL installation...ie: Ubuntu) - - zfs-local-ephemeral-inline (provisions node-local zfs datasets) + - `freenas-nfs` (manages zfs datasets to share over nfs) + - `freenas-iscsi` (manages zfs zvols to share over iscsi) + - `zfs-generic-nfs` (works with any ZoL installation...ie: Ubuntu) + - `zfs-generic-iscsi` (works with any ZoL installation...ie: Ubuntu) + - `zfs-local-ephemeral-inline` (provisions node-local zfs datasets) - framework for developing `csi` drivers If you have any interest in providing a `csi` driver, simply open an issue to @@ -25,18 +25,54 @@ relatively easy to implement new drivers. # Installation -Predominantly 2 things are needed: +Predominantly 3 things are needed: -- node prep: https://netapp-trident.readthedocs.io/en/stable-v20.04/kubernetes/operations/tasks/worker.html +- node prep +- server prep - deploy the driver into the cluster (`helm` chart provided with sample `values.yaml`) +## Node Prep + You should install/configure the requirements for both nfs and iscsi. +Follow the instructions here: https://netapp-trident.readthedocs.io/en/stable-v20.04/kubernetes/operations/tasks/worker.html + If you are running Kubernetes with rancher/rke please see the following: - https://github.com/rancher/rke/issues/1846 +### zfs-local-ephemeral-inline + +This `driver` provisions node-local ephemeral storage on a per-pod basis. Each +node should have an identically named zfs pool created and avaialble to the +`driver`. Note, this is _NOT_ the same thing as using the docker zfs storage +driver (although the same pool could be used). No other requirements are +necessary. + +- https://github.com/kubernetes/enhancements/blob/master/keps/sig-storage/20190122-csi-inline-volumes.md +- https://kubernetes-csi.github.io/docs/ephemeral-local-volumes.html + +## Server Prep + +Server preparation depends slightly on which `driver` you are using. + +### FreeNAS (freenas-nfs, freenas-iscsi) + +Ensure the following services are configurged and running: + +- ssh (if you use a password for authentication make sure it is allowed) +- nfs +- iscsi + +### ZoL (zfs-generic-nfs, zfs-generic-iscsi) + +Ensure ssh and zfs is installed on the server and that you have installed +`targetcli`. + +- `yum install targetcli -y` +- `apt-get -y install targetcli-fb` + ## Helm Installation ``` @@ -69,3 +105,30 @@ You may install multiple deployments of each/any driver. It requires the followi - Make sure you have a unique `csiDriver.name` in the values file - Use unqiue names for your storage classes (per cluster) - Use a unique parent dataset (ie: don't try to use the same parent across deployments or clusters) + +# Snapshot Support + +Install beta (v1.17+) CRDs (once per cluster): + +- https://github.com/kubernetes-csi/external-snapshotter/tree/master/config/crd + +``` +kubectl apply -f snapshot.storage.k8s.io_volumesnapshotclasses.yaml +kubectl apply -f snapshot.storage.k8s.io_volumesnapshotcontents.yaml +kubectl apply -f snapshot.storage.k8s.io_volumesnapshots.yaml +``` + +Install snapshot controller (once per cluster): + +- https://github.com/kubernetes-csi/external-snapshotter/tree/master/deploy/kubernetes/snapshot-controller + +``` +# replace namespace references to your liking +kubectl apply -f rbac-snapshot-controller.yaml +kubectl apply -f setup-snapshot-controller.yaml +``` + +Install `democratic-csi` as usual with `volumeSnapshotClasses` defined as appropriate. + +https://kubernetes.io/docs/concepts/storage/volume-snapshots/ +https://github.com/kubernetes-csi/external-snapshotter#usage diff --git a/src/driver/zfs-local-ephemeral-inline/index.js b/src/driver/zfs-local-ephemeral-inline/index.js index 208afb3..8b50cf1 100644 --- a/src/driver/zfs-local-ephemeral-inline/index.js +++ b/src/driver/zfs-local-ephemeral-inline/index.js @@ -330,6 +330,7 @@ class ZfsLocalEphemeralInlineDriver extends CsiBaseDriver { }); } + // TODO: catch out of space errors and return specifc grpc message? await zb.zfs.create(datasetName, { parents: true, properties: volumeProperties,