This section contains steps to install vSphere CSI Driver. Please visit the Prerequisite section before proceeding.
Note that this installation guide only applies to Vanilla Kubernetes clusters. Project Pacific and Tanzu Kubernetes Grid clusters come with vSphere CSI Driver pre installed.
The following steps need to be performed on the k8s node where the vSphere CSI drivers controller will be deployed. We recommend installing vSphere CSI driver on your k8s master node.
- Taint Master Node
- Create a configuration file with vSphere credentials
- Create a kubernetes secret for vSphere credentials
- Install vSphere CSI driver
- Verify that CSI has been successfully deployed
Make sure that the Master Node is tainted with
$ kubectl describe nodes | egrep "Taints:|Name:" Name: <k8s-master-name> Taints: node-role.kubernetes.io/master:NoSchedule Name: <k8s-worker1-name> Taints: <none> Name: <k8s-worker2-name> Taints: <none> Name: <k8s-worker3-name> Taints: <none> Name: <k8s-worker4-name> Taints: <none>
If it is not tainted, you may do so by running the command:
kubectl taint nodes <k8s-master-name> node-role.kubernetes.io/master=:NoSchedule
Create a configuration file that will contain details to connect to vSphere.
The default file to write these configuration details is the
csi-vsphere.conf file. If you would like to use a file with another name, change the environment variable
VSPHERE_CSI_CONFIG in the deployment YAMLs described in the section Install vSphere CSI driver below.
For deployment with zones refer to https://vsphere-csi-driver.sigs.k8s.io/driver-deployment/deploying_csi_with_zones.html
Here is an example vSphere configuration file for block volumes, with dummy values:
$ cat /etc/kubernetes/csi-vsphere.conf [Global] cluster-id = "<cluster-id>" cluster-distribution = "<cluster-distribution>" ca-file = <ca file path> # optional, use with insecure-flag set to false thumbprint = "<cert thumbprint>" # optional, use with insecure-flag set to false without providing ca-file [VirtualCenter "<IP or FQDN>"] insecure-flag = "<true or false>" user = "<username>" password = "<password>" port = "<port>" datacenters = "<datacenter1-path>, <datacenter2-path>, ..."
Where the entries have the following meaning:
cluster-id- represents the unique cluster identifier. Each kubernetes cluster should have it's own unique cluster-id set in the configuration file. The cluster ID should not exceed 64 characters.
cluster-distribution- represents the distribution of the kubernetes cluster. This parameter is optional but will be made mandatory in a future release. Examples are
values with special character
\rcauses vSphere CSI controller to go into CrashLoopBackOff state.
values of more than 128 characters will cause the PVC creation to be stuck in
VirtualCenter- section defines vCenter IP address / FQDN.
insecure-flag- should be set to true to use self-signed certificate for login.
user- vCenter username. Specify username along with domain name for example -
user = "userName@domainName"or
user = "domainName\\username". Note: For active directory user if domain name is not specified, vSphere CSI driver will not function properly.
password- password for vCenter user specified with user.
port- vCenter Server Port. The default is 443 if not specified.
ca-file- path to a CA certificate in PEM format. It is an optional parameter.
Thumbprint- the certificate thumbprint. It is an optional parameter and is ignored when you're using insecure setups or when you provide
datacenters- list of all comma separated datacenter paths where kubernetes node VMs are present. When datacenter is located at the root, the name of datacenter is enough but when datacenter is placed in the folder, path needs to be specified as
folder/datacenter-name. Please note since comma is used as a delimiter, the datacenter name itself must not contain a comma.
Note: To deploy CSI driver for block volume in VMC environment, in the vSphere configuration file, need to specifiy cloudadmin user in
user field and cloudadmin password in
For file volumes, there are some extra parameters added to the config to help specify network permissions and placement of volumes. A sample config file for file volumes is shown below.
$ cat /etc/kubernetes/csi-vsphere.conf [Global] cluster-id = "<cluster-id>" cluster-distribution = "<cluster-distribution>" ca-file = <ca file path> # optional, use with insecure-flag set to false [NetPermissions "A"] ips = "*" permissions = "READ_WRITE" rootsquash = false [NetPermissions "B"] ips = "10.20.20.0/24" permissions = "READ_ONLY" rootsquash = true [NetPermissions "C"] ips = "10.30.30.0/24" permissions = "NO_ACCESS" [NetPermissions "D"] ips = "10.30.10.0/24" rootsquash = true [NetPermissions "E"] ips = "10.30.1.0/24" [VirtualCenter "<IP or FQDN>"] insecure-flag = "<true or false>" user = "<username>" password = "<password>" port = "<port>" datacenters = "<datacenter1-path>, <datacenter2-path>, ..." targetvSANFileShareDatastoreURLs = "ds:///vmfs/volumes/vsan:52635b9067079319-95a7473222c4c9cd/" # Optional
Some of the parameters have been explained in the previous section for block volumes.
NetPermissions section are exclusive to file volumes and are optional.
targetvSANFileShareDatastoreURLs- optional parameter. It comes in handy when you have an environment with file service enabled vSAN cluster(s) and you intend to limit the creation of file share volumes to only a select few vSAN datastores. This field contains a comma separated list of datastore URLs where you want to deploy the file share volumes.
NetPermissions- optional parameter. Set of parameters used to restrict the network capabilities of all the file share volumes created under this vSphere configuration. If the complete set of
NetPermissionsare not mentioned for a given IP range, defaults are assumed for the missing parameters. You can define as many
NetPermissionssections as you want and each of these sections is uniquely identified by the string which follows.
The parameters grouped by
NetPermissions are as follows:
Ips- defines the IP range or IP subnet to which these restrictions will be levied upon. The default value for
Ipsis "*" which means all the IPs.
Permissions- can either be "READ_WRITE", "READ_ONLY" or "NO_ACCESS". The default value for
Permissionsis "READ_WRITE" for the given IP range.
RootSquash- defines the security access level for the file share volume. The default for
falsei.e allow root access to the all the file share volumes created within the given IP range.
NetPermissions section is completely omitted, the defaults for each of the parameters above are assumed.
Create a Kubernetes secret that will contain configuration details to connect to vSphere.
Create the secret by running the following command:
kubectl create secret generic vsphere-config-secret --from-file=csi-vsphere.conf --namespace=kube-system
Verify that the credential secret is successfully created in the kube-system namespace.
$ kubectl get secret vsphere-config-secret --namespace=kube-system NAME TYPE DATA AGE vsphere-config-secret Opaque 1 43s
For security purposes, it is advised to remove this configuration file.
Before you deploy the vSphere CSI driver, refer to the Compatibility page to view the supported kubernetes versions for a particular vSphere CSI version and feature support page to see what features are supported on that version.
Get the deployment manifests pertaining to the version of vSphere CSI driver.
- v2.2.0 deployment manifests - https://github.com/kubernetes-sigs/vsphere-csi-driver/tree/v2.2.0/manifests/v2.2.0
- v2.1.1 deployment manifests - https://github.com/kubernetes-sigs/vsphere-csi-driver/tree/v2.1.1/manifests/v2.1.1
- v2.1.0 deployment manifests - https://github.com/kubernetes-sigs/vsphere-csi-driver/tree/release-2.1/manifests/v2.1.0
NOTE: Refer vSphere CSI Driver - Deployment with Topology to deploy your kubernetes cluster with topology aware provisioning feature.
Create the roles, cluster roles and service accounts needed for installation of vSphere CSI Driver by deploying the YAML files available in the
rbacfolder of the vSphere CSI driver version you have chosen.
$ kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/vsphere-csi-driver/v2.2.0/manifests/v2.2.0/rbac/vsphere-csi-controller-rbac.yaml $ kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/vsphere-csi-driver/v2.2.0/manifests/v2.2.0/rbac/vsphere-csi-node-rbac.yaml
Deploy the CSI controller and node daemonset using the YAML files available in the
deployfolder of the same version.
$ kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/vsphere-csi-driver/v2.2.0/manifests/v2.2.0/deploy/vsphere-csi-controller-deployment.yaml $ kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/vsphere-csi-driver/v2.2.0/manifests/v2.2.0/deploy/vsphere-csi-node-ds.yaml
To verify that the CSI driver has been successfully deployed, you should observe that there is one instance of the vsphere-csi-controller running on the master node and that an instance of the vsphere-csi-node is running on each of the worker nodes.
$ kubectl get deployment --namespace=kube-system NAME READY AGE vsphere-csi-controller 1/1 2m58s $ kubectl get daemonsets vsphere-csi-node --namespace=kube-system NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE vsphere-csi-node 4 4 4 4 4 <none> 3m51s
Verify that the vSphere CSI driver has been registered with Kubernetes
$ kubectl describe csidrivers Name: csi.vsphere.vmware.com Namespace: Labels: <none> Annotations: <none> API Version: storage.k8s.io/v1beta1 Kind: CSIDriver Metadata: Creation Timestamp: 2020-04-14T20:46:07Z Resource Version: 2382881 Self Link: /apis/storage.k8s.io/v1beta1/csidrivers/csi.vsphere.vmware.com UID: 19afbecd-bc2f-4806-860f-b29e20df3074 Spec: Attach Required: true Pod Info On Mount: false Volume Lifecycle Modes: Persistent Events: <none>
Verify that the CSINodes have been created
$ kubectl get CSINode NAME CREATED AT <k8s-worker1-name> 2020-04-14T12:30:29Z <k8s-worker2-name> 2020-04-14T12:30:38Z <k8s-worker3-name> 2020-04-14T12:30:21Z <k8s-worker4-name> 2020-04-14T12:30:26Z