documentation additions

README.md

@@ -6,17 +6,17 @@ providing storage for various container orchestration systems (ie: Kubernetes).
 The current focus is providing storage via iscsi/nfs from zfs-based storage
 systems, predominantly `FreeNAS / TrueNAS` and `ZoL` on `Ubuntu`.
 
-The current drivers implement depth and breadth of the `csi` spec, so you have
-access to resizing, snapshots, etc, etc.
+The current drivers implement the depth and breadth of the `csi` spec, so you
+have access to resizing, snapshots, clones, etc functionality.
 
 `democratic-csi` is 2 things:
 
 - several implementations of `csi` drivers
-  - freenas-nfs (manages zfs datasets to share over nfs)
-  - freenas-iscsi (manages zfs zvols to share over iscsi)
-  - zfs-generic-nfs (works with any ZoL installation...ie: Ubuntu)
-  - zfs-generic-iscsi (works with any ZoL installation...ie: Ubuntu)
-  - zfs-local-ephemeral-inline (provisions node-local zfs datasets)
+  - `freenas-nfs` (manages zfs datasets to share over nfs)
+  - `freenas-iscsi` (manages zfs zvols to share over iscsi)
+  - `zfs-generic-nfs` (works with any ZoL installation...ie: Ubuntu)
+  - `zfs-generic-iscsi` (works with any ZoL installation...ie: Ubuntu)
+  - `zfs-local-ephemeral-inline` (provisions node-local zfs datasets)
 - framework for developing `csi` drivers
 
 If you have any interest in providing a `csi` driver, simply open an issue to
@@ -25,18 +25,54 @@ relatively easy to implement new drivers.
 
 # Installation
 
-Predominantly 2 things are needed:
+Predominantly 3 things are needed:
 
-- node prep: https://netapp-trident.readthedocs.io/en/stable-v20.04/kubernetes/operations/tasks/worker.html
+- node prep
+- server prep
 - deploy the driver into the cluster (`helm` chart provided with sample
   `values.yaml`)
 
+## Node Prep
+
 You should install/configure the requirements for both nfs and iscsi.
 
+Follow the instructions here: https://netapp-trident.readthedocs.io/en/stable-v20.04/kubernetes/operations/tasks/worker.html
+
 If you are running Kubernetes with rancher/rke please see the following:
 
 - https://github.com/rancher/rke/issues/1846
 
+### zfs-local-ephemeral-inline
+
+This `driver` provisions node-local ephemeral storage on a per-pod basis. Each
+node should have an identically named zfs pool created and avaialble to the
+`driver`. Note, this is _NOT_ the same thing as using the docker zfs storage
+driver (although the same pool could be used). No other requirements are
+necessary.
+
+- https://github.com/kubernetes/enhancements/blob/master/keps/sig-storage/20190122-csi-inline-volumes.md
+- https://kubernetes-csi.github.io/docs/ephemeral-local-volumes.html
+
+## Server Prep
+
+Server preparation depends slightly on which `driver` you are using.
+
+### FreeNAS (freenas-nfs, freenas-iscsi)
+
+Ensure the following services are configurged and running:
+
+- ssh (if you use a password for authentication make sure it is allowed)
+- nfs
+- iscsi
+
+### ZoL (zfs-generic-nfs, zfs-generic-iscsi)
+
+Ensure ssh and zfs is installed on the server and that you have installed
+`targetcli`.
+
+- `yum install targetcli -y`
+- `apt-get -y install targetcli-fb`
+
 ## Helm Installation
 
 ```
@@ -69,3 +105,30 @@ You may install multiple deployments of each/any driver. It requires the followi
 - Make sure you have a unique `csiDriver.name` in the values file
 - Use unqiue names for your storage classes (per cluster)
 - Use a unique parent dataset (ie: don't try to use the same parent across deployments or clusters)
+
+# Snapshot Support
+
+Install beta (v1.17+) CRDs (once per cluster):
+
+- https://github.com/kubernetes-csi/external-snapshotter/tree/master/config/crd
+
+```
+kubectl apply -f snapshot.storage.k8s.io_volumesnapshotclasses.yaml
+kubectl apply -f snapshot.storage.k8s.io_volumesnapshotcontents.yaml
+kubectl apply -f snapshot.storage.k8s.io_volumesnapshots.yaml
+```
+
+Install snapshot controller (once per cluster):
+
+- https://github.com/kubernetes-csi/external-snapshotter/tree/master/deploy/kubernetes/snapshot-controller
+
+```
+# replace namespace references to your liking
+kubectl apply -f rbac-snapshot-controller.yaml
+kubectl apply -f setup-snapshot-controller.yaml
+```
+
+Install `democratic-csi` as usual with `volumeSnapshotClasses` defined as appropriate.
+
+https://kubernetes.io/docs/concepts/storage/volume-snapshots/
+https://github.com/kubernetes-csi/external-snapshotter#usage

src/driver/zfs-local-ephemeral-inline/index.js

@@ -330,6 +330,7 @@ class ZfsLocalEphemeralInlineDriver extends CsiBaseDriver {
       });
     }
 
+    // TODO: catch out of space errors and return specifc grpc message?
     await zb.zfs.create(datasetName, {
       parents: true,
       properties: volumeProperties,