vSphere CSI Driver - Installation
This section contains steps to install vSphere CSI Driver. Please visit the Prerequisite section before proceeding.
Note that this installation guide only applies to Vanilla Kubernetes clusters. Project Pacific and Tanzu Kubernetes Grid clusters come with vSphere CSI Driver pre installed.
The following steps need to be performed on the k8s node where the vSphere CSI drivers controller will be deployed. We recommend installing vSphere CSI driver on your k8s master node.
- Taint Master Node
- Create a configuration file with vSphere credentials
- Create a kubernetes secret for vSphere credentials
- Create Roles, ServiceAccount and ClusterRoleBinding for vSphere CSI Driver
- Install vSphere CSI driver
- Verify that CSI has been successfully deployed
Make sure that the Master Node is tainted with
$ kubectl describe nodes | egrep "Taints:|Name:" Name: <k8s-master-name> Taints: node-role.kubernetes.io/master:NoSchedule Name: <k8s-worker1-name> Taints: <none> Name: <k8s-worker2-name> Taints: <none> Name: <k8s-worker3-name> Taints: <none> Name: <k8s-worker4-name> Taints: <none>
If it is not tainted, you may do so by running the command:
kubectl taint nodes <k8s-master-name> node-role.kubernetes.io/master=:NoSchedule
Create a configuration file that will contain details to connect to vSphere.
The default file to write these configuration details is the
csi-vsphere.conf file. If you would like to use a file with another name, change the environment variable
VSPHERE_CSI_CONFIG in the deployment YAMLs described in the section Install vSphere CSI driver below.
Here is an example vSphere configuration file for block volumes, with dummy values:
$ cat /etc/kubernetes/csi-vsphere.conf [Global] cluster-id = "<cluster-id>" ca-file = <ca file path> # optional, use with insecure-flag set to false thumbprint = "<cert thumbprint>" # optional, use with insecure-flag set to false without providing ca-file [VirtualCenter "<IP or FQDN>"] insecure-flag = "<true or false>" user = "<username>" password = "<password>" port = "<port>" datacenters = "<datacenter1-path>, <datacenter2-path>, ..."
Where the entries have the following meaning:
cluster-id- represents the unique cluster identifier. Each kubernetes cluster should have it's own unique cluster-id set in the configuration file. The cluster ID should not exceed 64 characters.
VirtualCenter- section defines vCenter IP address / FQDN.
insecure-flag- should be set to true to use self-signed certificate for login.
user- vCenter username.
password- password for vCenter user specified with user.
port- vCenter Server Port. The default is 443 if not specified.
ca-file- path to a CA certificate in PEM format. It is an optional parameter.
Thumbprint- the certificate thumbprint. It is an optional parameter and is ignored when you're using insecure setups or when you provide
datacenters- list of all comma separated datacenter paths where kubernetes node VMs are present. When datacenter is located at the root, the name of datacenter is enough but when datacenter is placed in the folder, path needs to be specified as
folder/datacenter-name. Please note since comma is used as a delimiter, the datacenter name itself must not contain a comma.
For file volumes, there are some extra parameters added to the config to help specify network permissions and placement of volumes. A sample config file for file volumes is shown below.
$ cat /etc/kubernetes/csi-vsphere.conf [Global] cluster-id = "<cluster-id>" ca-file = <ca file path> # optional, use with insecure-flag set to false [NetPermissions "A"] ips = "*" permissions = "READ_WRITE" rootsquash = false [NetPermissions "B"] ips = "10.20.20.0/24" permissions = "READ_ONLY" rootsquash = true [NetPermissions "C"] ips = "10.30.30.0/24" permissions = "NO_ACCESS" [NetPermissions "D"] ips = "10.30.10.0/24" rootsquash = true [NetPermissions "E"] ips = "10.30.1.0/24" [VirtualCenter "<IP or FQDN>"] insecure-flag = "<true or false>" user = "<username>" password = "<password>" port = "<port>" datacenters = "<datacenter1-path>, <datacenter2-path>, ..." targetvSANFileShareDatastoreURLs = "ds:///vmfs/volumes/vsan:52635b9067079319-95a7473222c4c9cd/" # Optional
Some of the parameters have been explained in the previous section for block volumes.
NetPermissions section are exclusive to file volumes and are optional.
targetvSANFileShareDatastoreURLs- optional parameter. It comes in handy when you have an environment with file service enabled vSAN cluster(s) and you intend to limit the creation of file share volumes to only a select few vSAN datastores. This field contains a comma separated list of datastore URLs where you want to deploy the file share volumes.
NetPermissions- optional parameter. Set of parameters used to restrict the network capabilities of all the file share volumes created under this vSphere configuration. If the complete set of
NetPermissionsare not mentioned for a given IP range, defaults are assumed for the missing parameters. You can define as many
NetPermissionssections as you want and each of these sections is uniquely identified by the string which follows.
The parameters grouped by
NetPermissions are as follows:
Ips- defines the IP range or IP subnet to which these restrictions will be levied upon. The default value for
Ipsis "*" which means all the IPs.
Permissions- can either be "READ_WRITE", "READ_ONLY" or "NO_ACCESS". The default value for
Permissionsis "READ_WRITE" for the given IP range.
RootSquash- defines the security access level for the file share volume. The default for
falsei.e allow root access to the all the file share volumes created within the given IP range.
NetPermissions section is completely omitted, the defaults for each of the parameters above are assumed.
Create a Kubernetes secret that will contain configuration details to connect to vSphere.
Create the secret by running the following command:
kubectl create secret generic vsphere-config-secret --from-file=csi-vsphere.conf --namespace=kube-system
Verify that the credential secret is successfully created in the kube-system namespace.
$ kubectl get secret vsphere-config-secret --namespace=kube-system NAME TYPE DATA AGE vsphere-config-secret Opaque 1 43s
For security purposes, it is advised to remove this configuration file.
Create ClusterRole, ServiceAccounts and ClusterRoleBinding needed for installation of vSphere CSI Driver
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/vsphere-csi-driver/master/manifests/v2.0.0/vsphere-7.0/vanilla/rbac/vsphere-csi-controller-rbac.yaml
Click here to view the roles assigned to vSphere CSI Driver.
Our CSI Controller runs as a Kubernetes deployment, with a replica count of 1. For version
vsphere-csi-controller Pod consists of 6 containers – the CSI controller, External Provisioner, External Attacher, External Resizer, Liveness probe and vSphere Syncer.
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/vsphere-csi-driver/master/manifests/v2.0.0/vsphere-7.0/vanilla/deploy/vsphere-csi-controller-deployment.yaml
There is also a CSI node Daemonset to be deployed, that will run on every node.
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/vsphere-csi-driver/master/manifests/v2.0.0/vsphere-7.0/vanilla/deploy/vsphere-csi-node-ds.yaml
Click here to view the deployment manifest for vSphere CSI driver.
NOTE: Visit vSphere CSI Driver - Deployment with Topology to deploy your kubernetes cluster with topology aware provisioning feature.
To verify that the CSI driver has been successfully deployed, you should observe that there is one instance of the vsphere-csi-controller running on the master node and that an instance of the vsphere-csi-node is running on each of the worker nodes.
$ kubectl get deployment --namespace=kube-system NAME READY AGE vsphere-csi-controller 1/1 2m58s $ kubectl get daemonsets vsphere-csi-node --namespace=kube-system NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE vsphere-csi-node 4 4 4 4 4 <none> 3m51s
Verify that the vSphere CSI driver has been registered with Kubernetes
$ kubectl describe csidrivers Name: csi.vsphere.vmware.com Namespace: Labels: <none> Annotations: <none> API Version: storage.k8s.io/v1beta1 Kind: CSIDriver Metadata: Creation Timestamp: 2020-04-14T20:46:07Z Resource Version: 2382881 Self Link: /apis/storage.k8s.io/v1beta1/csidrivers/csi.vsphere.vmware.com UID: 19afbecd-bc2f-4806-860f-b29e20df3074 Spec: Attach Required: true Pod Info On Mount: false Volume Lifecycle Modes: Persistent Events: <none>
Verify that the CSINodes have been created
$ kubectl get CSINode NAME CREATED AT <k8s-worker1-name> 2020-04-14T12:30:29Z <k8s-worker2-name> 2020-04-14T12:30:38Z <k8s-worker3-name> 2020-04-14T12:30:21Z <k8s-worker4-name> 2020-04-14T12:30:26Z