Nfs-client-provisioner是一个开源的NFS 外部Provisioner，利用NFS Server为Kubernetes集群提供持久化存储。部署nfs-client-provisioner后，您可以在Kubernetes集群中动态创建PV。
Kubernetes 命令行客户端 kubectl可以让您从客户端计算机连接到 Kubernetes 集群，实现应用部署。详情参考使用Kubectl客户端连接到Kubernetes集群。
详情参考在集群中部署nfs-client-provisioner。
StorageClass为PVC提供动态发现、绑定PV的配置。因此创建一个使用nfs-client-provisioner的Storage Class，并在创建PVC时显式指定对应的StorageClassName，就可以基于CFS云文件存储在Kubernetes集群中动态创建NFS类型的PV。
wget https://kubernetes.s3.cn-north-1.jdcloud-oss.com/CFS/storageClass-With-jdcloud-cfs.yml
apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: auto-cfs-storage provisioner: jdcloud-cfs #nfs-client-provisioner deployment env中定义的PROVISIONER_NAME保持一致 mountOptions: - vers=3 - noresvport parameters: mountOptions: "vers=3,noresvport" archiveOnDelete: "false" #archiveOnDelete定义为false时，删除NFS Server中对应的目录，为true则保留；
备注：CFS目前暂不支持flock，如果应用需要flock支持，请开工单单独开通. 如果NFS需要支持子目录，请将vers=3变更为vers=4
kubectl create -f storageClass-With-jdcloud-cfs.yml
kubectl get storageClass auto-cfs-storage NAME PROVISIONER AGE auto-cfs-storage jdcloud-cfs 5m15s
wget https://kubernetes.s3.cn-north-1.jdcloud-oss.com/CFS/Create-PV-dynamically.yml
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: auto-pv-with-nfs-client-provisioner
spec:
storageClassName: auto-cfs-storage #使用nfs-client-provisioner作为NFS外部provisioner的Storage Class Name；
accessModes:
- ReadWriteMany
resources:
requests:
storage: 1Mi #NFS Server中对应的挂载目录大小；目前CFS文件存储不限制挂载目录的容量；storage不超过文件存储最大容量限制即可
kubectl create -f Create-PV-dynamically.yml
kubectl get pvc auto-pv-with-nfs-client-provisioner
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
auto-pv-with-nfs-client-provisioner Bound pvc-c44da35f-b8bc-11e9-b6cc-fa163e229fe7 1Mi RWX auto-cfs-storage 10s
kubectl describe pvc auto-pv-with-nfs-client-provisioner
Name: auto-pv-with-nfs-client-provisioner
Namespace: default
StorageClass: auto-cfs-storage
Status: Bound
Volume: pvc-c44da35f-b8bc-11e9-b6cc-fa163e229fe7 #根据Storage Class配置动态创建的PV名称
Labels: <none>
Annotations: pv.kubernetes.io/bind-completed: yes
pv.kubernetes.io/bound-by-controller: yes
volume.beta.kubernetes.io/storage-provisioner: jdcloud-cfs
Finalizers: [kubernetes.io/pvc-protection]
Capacity: 1Mi
Access Modes: RWX
Events:
kubectl get pv pvc-c44da35f-b8bc-11e9-b6cc-fa163e229fe7
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pvc-c44da35f-b8bc-11e9-b6cc-fa163e229fe7 1Mi RWX Delete Bound default/auto-pv-with-nfs-client-provisioner auto-cfs-storage 15m
kubectl describe pv pvc-c44da35f-b8bc-11e9-b6cc-fa163e229fe7
Name: pvc-c44da35f-b8bc-11e9-b6cc-fa163e229fe7
Labels: <none>
Annotations: pv.kubernetes.io/provisioned-by: jdcloud-cfs
Finalizers: [kubernetes.io/pv-protection]
StorageClass: auto-cfs-storage
Status: Bound
Claim: default/auto-pv-with-nfs-client-provisioner
Reclaim Policy: Delete
Access Modes: RWX
Capacity: 1Mi
Node Affinity: <none>
Message:
Source:
Type: NFS (an NFS mount that lasts the lifetime of a pod)
Server: 10.XX.XX.11 #CFS文件存储中对应的挂载目标IP地址
Path: /cfs/default-auto-pv-with-nfs-client-provisioner-pvc-c44da35f-b8bc-11e9-b6cc-fa163e229fe7 # NFS Server中由nfs-client-provisioner自动创建的子目录
ReadOnly: false
Events: <none>
sudo yum install –y nfs-utils
mkdir /cfs #在云主机上创建一个新目录 mount -t nfs 172.XX.XX.10:/cfs /cfs #将nfs 172.XX.XX.10:/cfs挂载到云主机的cfs目录上，其中172.XX.XX.10请使用云文件存储的挂载目标IP替换
cd /cfs #进入云主机上的cfs目录 ls -a # 查看cfs目录中的内容 default-auto-pv-with-nfs-client-provisioner-pvc-c44da35f-b8bc-11e9-b6cc-fa163e229fe7 #ls输出内容，验证cfs目录下新增了子目录，子目录名称与PV Source.Path一致的目录
wget https://kubernetes.s3.cn-north-1.jdcloud-oss.com/CFS/pod-touch-cfs.yml
kind: Pod
apiVersion: v1
metadata:
name: pod-touch-cfs
spec:
containers:
- name: test-pod
image: busybox
imagePullPolicy: IfNotPresent
command:
- "/bin/sh"
args:
- "-c"
- "touch /mnt/SUCCESS && exit 0 || exit 1" #在/mnt目录下新建一个名称为SUCCESS的文件，并写入helloworld
volumeMounts:
- name: nfs-pvc
mountPath: "/mnt" #将volume挂载到Pod的/mnt目录
restartPolicy: "Never"
volumes:
- name: nfs-pvc
persistentVolumeClaim:
claimName: auto-pv-with-nfs-client-provisioner #指定与云文件存储建立绑定关系的PVC 名称
kubectl create -f pod-touch-cfs.yml
kubectl get pod pod-touch-cfs NAME READY STATUS RESTARTS AGE pod-touch-cfs 1/1 Running 0 11s 执行exec进入pod，验证文件内容 kubectl exec -it pod-touch-cfs /bin/sh / # cat /mnt/SUCCESS helloworld
cd /cfs #进入云主机上的cfs目录 ls -a # 查看cfs目录中的内容 default-auto-pv-with-nfs-client-provisioner-pvc-c44da35f-b8bc-11e9-b6cc-fa163e229fe7 #ls输出内容，验证cfs目录下新增了子目录，子目录名称与PV Source.Path一致的目录 cat default-auto-pv-with-nfs-client-provisioner-pvc-c44da35f-b8bc-11e9-b6cc-fa163e229fe7/SUCCESS helloworld #与PV Source.Path一致的子目录下，查看新增文件SUCCESS的内容
kubectl delete pod pod-touch-cfs pod "pod-touch-cfs" deleted
wget https://kubernetes.s3.cn-north-1.jdcloud-oss.com/CFS/verify-pv-cfs.yml
kind: Pod
apiVersion: v1
metadata:
name: verify-pv-cfs
spec:
containers:
- name: c1
image: busybox
imagePullPolicy: IfNotPresent
command:
- /bin/sh
args:
- -c
- 'while true; do ls -l /mnt/cfs/; sleep 2; done'
volumeMounts:
- mountPath: "/mnt/cfs"
name: cfs-pv001
volumes:
- name: cfs-pv001
persistentVolumeClaim:
claimName: auto-pv-with-nfs-client-provisioner #与云文件存储建立绑定关系的PVC 名称
kubectl create -f verify-pv-cfs.yml pod/verify-pv-cfs created
kubectl get pod verify-pv-cfs NAME READY STATUS RESTARTS AGE verify-pv-cfs 1/1 Running 0 15s 查看Pod verify-pv-cfs输出的日志： kubectl logs verify-pv-cfs total 1 -rw-r--r-- 1 root root 18 Aug 7 07:29 SUCCESS exec进入Pod verify-pv-cfs，查看SUCCESS文件的内容： kubectl exec -it verify-pv-cfs /bin/sh / # cat /mnt/cfs/SUCCESS helloworld
kubectl delete pod verify-pv-cfs pod "verify-pv-cfs" deleted
kubectl delete pvc auto-pv-with-nfs-client-provisioner persistentvolumeclaim "auto-pv-with-nfs-client-provisioner" deleted
kubectl get pv pvc-c44da35f-b8bc-11e9-b6cc-fa163e229fe7 Error from server (NotFound): persistentvolumes "pvc-c44da35f-b8bc-11e9-b6cc-fa163e229fe7" not found 输出以上结果，说明与PVC关联的PV pvc-c44da35f-b8bc-11e9-b6cc-fa163e229fe7已被自动删除
cd /cfs #进入云主机上的cfs目录 ls -a # 查看cfs目录中的内容 输出内容说明：输出内容为空，则表明PVC、PV被删除后，CFS文件存储中由nfs-client-provisioner自动创建的子目录及文件内容已被成功回收