public_git
/
democratic-csi
mirror of https://github.com/democratic-csi/democratic-csi.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
democratic-csi/src/driver/controller-proxy/compatibility.md

4.6 KiB
Raw Blame History

Proxy driver compatibility

There are 2 challenges with the proxy driver:

  1. Proxy needs dynamic state. CSI spec implies that dynamic state must be external, which isn't ideal for small deployments, and is incompatible with democratic-csi.

  2. Proxy must provide a common set of capabilities for all drivers it represents.

A great discussion of difficulties per storage class state can be found here:

This file contains implementation details relevant for development. If you are searching for user documentation and deployment example go here.

Terminology and structure

"Proxy" is the driver created via driver: proxy in the main config. Other drivers are referred to as "real driver" and "underlying driver".

"Connection" is a way to distinguish real drivers in proxy driver calls

  • Connection name is set in storage class parameters
  • Connection name is stored in volume handle
  • Connection name is used as part of config file path

All config files must be mounted into democratic-csi filesystem. They can be added, updated and removed dynamically.

CSI features

Generally most features are supported.

However, some calls will not work:

NodeGetInfo works but it brings additional challenges. Node info is common for all storage classes. If different drivers need different output in NodeGetInfo, they can't coexist. See node info support notes

Driver compatibility

Proxy driver has the following minimal requirements for real underlying drivers:

  • Node methods should not use config values
    • This can be lifted
    • This is added because drivers use only volume context for mounting, and sometimes secrets
      • There is one exception to this rule, and I would argue that that driver is just broken
  • Driver should not need any exotic capabilities, since capabilities are shared
  • Driver should use CreateVolume, so that proxy can set proper volume_id
  • Controller publishing is not supported, see Controller publish support

Proxy advertises that it supports most CSI methods. If some methods are missing from underlying driver, proxy will throw INVALID_ARGUMENT error. Some methods are expected to be missing from some of the underlying drivers. In such cases proxy returns default value:

  • GetCapacity returns infinite capacity when underlying driver does not report capacity

Volume ID format

  • volume_id format: v:connection-name/original-handle
  • snapshot_id format: s:connection-name/original-handle

Where:

  • v, s - fixed prefix
    • Allows to check that volume ID was created using proxy driver
  • connection-name - identifies connection for all CSI calls
  • original-handle - volume_id handle created by the underlying driver

Controller publish support

ControllerPublishVolume is not implemented because currently no driver need this. Implementation would need to replace node_id just like other methods replace volume_id.

See node info support notes

Incompatible drivers

  • zfs-local-ephemeral-inline: proxy can't set volume_id in CreateVolume to identify underlying driver
    • are inline-ephemeral and standard drivers even compatible?
  • objectivefs: NodeStageVolume uses driver parameters
    • NodeStageVolume needs this.options in getDefaultObjectiveFSInstance
    • Other node methods don't need driver options
    • Possible fix: add support for config values for node methods
    • Possible fix: add public pool data into volume attributes, move private data (if any) into a secret

Volume cloning and snapshots

Cloning works without any adjustments when both volumes use the same connection. If the connection is different:

  • TODO: Same driver, same server
    • It's up to driver to add support
    • Support is easy: just need to get proper source location in the CreateVolume
  • TODO: Same driver, different servers
    • It's up to driver to add support
    • Example: zfs send-receive
    • Example: file copy between nfs servers
  • Different drivers: block <-> file: unlikely to be practical
    • Users should probably do such things manually, by mounting both volumes into a pod
  • Different drivers: same filesystem type
    • Drivers should implement generic export and import functions
    • For example: TrueNas -> generic-zfs can theoretically be possible via zfs send
    • For example: nfs -> nfs can theoretically be possible via file copy
    • How to coordinate different drivers?