Introduction to the Container Storage Interface (CSI) of Kubernetes

Introduction to the Container Storage Interface (CSI) of Kubernetes

Container Storage Interface (CSI) is a function proposed at Kubernetes version 1.9 to solve the following issues.

The existing K8s volume plugin is in-tree, which means the source code of the plugin is under K8s’repository. This makes plugin need to send pull request to the K8s project before merging the commits. Also, the plugin will be compiled into executable files with K8s repository. It causes the update of the plugin slow, and third-party volume plugin developer will be required when developing the K8s.

A specific volume plugin will be asked to install on K8s nodes. To use Block Device in Ceph as the volume on K8s, we need to install ceph-common on the node before mounting the RBD image. This asks us to type the installing script into deployment tool before generating a K8s cluster. Or, connecting to the node to install when needed. The ideal case is the dependency of plugin can be combined into the container. It will have the ability to mount the volume when creating the Pod, and we do not need to worry about the dependency.

A single-node hostPath CSI driver

Before using CSI, we need to check if the flag “–allow-privileged=true” is inserted when initiating kubelet and api-server. This makes the Pod can be ran under privileged mode.

1.Create Pod

$ kubectl create -f https://raw.githubusercontent.com/kubernetes-csi/docs/master/book/src/example/csi-setup.yaml
storageclass "csi-hostpath-sc" created
serviceaccount "csi-service-account" created
clusterrole "csi-cluster-role" created
clusterrolebinding "csi-role-binding" created
pod "csi-pod" created

As we can see in YMAL, besides the hostPath CSI driver, there still have three sidecar container provided officially: external-attacher, external-provisioner, and driver-registrar. Their functionalities are listed below:

  • external-attacher

The external-attacher is an external controller that monitors VolumeAttachment objects created by controller-manager and attaches/detaches volumes to/from nodes (i.e. calls ControllerPublish/ControllerUnpublish. It is not ran on hostPath CSI driver.

  • external-provisioner

The external-provisioner is an external controller that monitors PersistentVolumeClaim objects created by user and creates/deletes volumes for them. In hostPath CSI driver, CreateVolume creates the required folder; while DeleteVolume deletes the specific folder on the node which driver is located.

  • driver-registrar

Adds the drivers custom NodeId (retrieved via GetNodeID call) to an annotation on the Kubernetes Node API Object.

$ kubectl describe no/k8s-01
...
Annotations:        csi.volume.kubernetes.io/nodeid={"csi-hostpath":"k8s-01"}
...

2.Create PersistentVolumeClaim

$ kubectl create -f https://raw.githubusercontent.com/kubernetes-csi/docs/master/book/src/example/csi-pvc.yaml
persistentvolumeclaim "csi-pvc" created
$ kubectl get pvc
NAME      STATUS    VOLUME                 CAPACITY   ACCESS MODES   STORAGECLASS      AGE
csi-pvc   Bound     pvc-c84b83e188d711e8   1Gi        RWO            csi-hostpath-sc   6s

Create PV dynamically after creating PVC

$ kubectl get pv
NAME                   CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS    CLAIM             STORAGECLASS      REASON    AGE
pvc-c84b83e188d711e8   1Gi        RWO            Delete           Bound     default/csi-pvc   csi-hostpath-sc             4s

When PVC object is created, external provisioner will create a folder under /tmp in hostpath-driver container.

$ kubectl exec -it csi-pod -c hostpath-driver -- /bin/sh
$ ls -l /tmp
total 4
drwxr-xr-x    2 root     root          4096 Jul 16 09:08 c85f4370-88d7-11e8-b7ad-0a580ae94006

3.Create Pod to mount PVC

$ kubectl create -f https://raw.githubusercontent.com/kubernetes-csi/docs/master/book/src/example/csi-app.yaml
pod "my-csi-app" created

4.Write files into volume and check if they are in the hostpath-driver container.

$ kubectl exec -it my-csi-app /bin/sh
/ # touch /data/test.txt
$ kubectl exec -it csi-pod -c hostpath-driver -- /bin/sh
/ # cd /tmp/c85f4370-88d7-11e8-b7ad-0a580ae94006/
/tmp/c85f4370-88d7-11e8-b7ad-0a580ae94006 # ls
test.txt

References

  1. Introducing Container Storage Interface (CSI) Alpha for Kubernetes  – Kubernetes
  2. Introduction – Kubernetes CSI Documentation

By:  杜永鴻 迎棧科技工程師

EDM

Select list(s)*

 

Loading