虚拟节点部署

前提条件

• 已创建JKE托管集群，请参见创建集群；

• 已通过kubectl工具连接集群。具体操作，请参见连接集群；

部署virtual node

说明：
1）虚拟节点与可用区绑定，若Serverless Pod需要调度至多个可用区，则每个可用区各需要部署至少一个虚拟节点。
2）部署虚拟节点时，可指定调度到虚拟节点上Pod底层使用的原生容器规格/规格族。如：

• 指定Pod底层使用 通用标准型-三代机：g.n3

• 指定Pod底层使用 通用标准型，不区分代次：g

• 指定Pod底层使用 通用算力型：u

1）权限生成
vk-sa.yaml文件内容如下，使用kubectl应用创建ServiceAccout、ClusterRoleBinding资源。

apiVersion: v1
kind: ServiceAccount
metadata:
  name: virtual-kubelet
  namespace: kube-system
  labels:
    k8s-app: virtual-kubelet
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: virtual-kubelet 
  namespace: kube-system
subjects:
- kind: ServiceAccount
  name: virtual-kubelet 
  namespace: kube-system 
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin

2）创建虚拟节点

vk-node.yaml模板及自定义字段说明如下，使用kubectl应用创建Secret、Deployment、CSINode等资源。

apiVersion: v1
kind: Secret
metadata:
  name: vk-secret
  namespace: kube-system
data:
  ACCESS_KEY: xxx # ak base64
  SECRET_KEY: xxx # sk base64
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: virtual-kubelet-cn-north-1b # vk node节点名称
  namespace: kube-system
  labels:
    k8s-app: kubelet
spec:
  replicas: 1
  selector:
    matchLabels:
      k8s-app: virtual-kubelet-cn-north-1b # vk node节点名称
  template:
    metadata:
      labels:
        k8s-app: virtual-kubelet-cn-north-1b # vk node节点名称
    spec:
      initContainers:
      - name: init-config 
        image: jdcloud-cn-north-1.jcr.service.jdcloud.com/virtual-kubelet:v1.18.10
        imagePullPolicy: Always
        envFrom:
        - secretRef:
            name: vk-secret
        env:
        - name: REGION
          value: cn-north-1 # 地域
        - name: AVALIABILITY_ZONE
          value: cn-north-1b # 可用区
        - name: CLUSTER_ID # 集群id
          value: cluster-fjbfrdfvz2sj
        - name: VPC_ID # vpc_id
          value: vpc-3nulk90wvb
        - name: SUBNET_ID # subnetid
          value: subnet-0yvjeyg1t4
        - name: SG_IDS # sgId
          value: sg-wej3v1ksan
        - name: DNS_NAMESERVERS # dns-nameserver
          value: "172.16.0.10"
        - name: NCOPENAPI_ADDR
          value: pod.jdcloud-api.com
        - name: NCOPENAPI_SCHEME
          value: https
        - name: NCOPENAPI_TIMEOUT
          value: "10000"
        - name: VMOPENAPI_ADDR
          value: vm.jdcloud-api.com
        - name: VMOPENAPI_SCHEME
          value: https
        - name: VMOPENAPI_TIMEOUT
          value: "10000"
        - name: METRICOPENAPI_ADDR
          value: monitor.jdcloud-api.com
        - name: METRICOPENAPI_SCHEME
          value: https
        - name: METRICOPENAPI_TIMEOUT
          value: "10000"
        - name: ZFS_OPENAPI_ADDR
          value: cfs.jdcloud-api.com
        - name: ZFS_OPENAPI_SCHEME
          value: https
        - name: ZFS_OPENAPI_TIMEOUT
          value: "10000"
        command: ["/bin/sh"]
        args: ["-c","init-config.sh u"] # u为通用算力型实例标识
        volumeMounts:
        - name: configs
          mountPath: "/etc/virtual-kubelet/config"
          readOnly: false 
      containers:
      - name: virtual-kubelet
        image: jdcloud-cn-north-1.jcr.service.jdcloud.com/virtual-kubelet:v1.18.10
        imagePullPolicy: Always
        envFrom:
        - secretRef:
             name: vk-secret
        env:
        - name: KUBERNETES_SERVICE_HOST # k8s service ip
          value: 172.16.0.1
        - name: KUBERNETES_SERVICE_PORT
          value: "443"
        - name: KUBELET_PORT
          value: "10250"
        - name: DEFAULT_NODE_NAME
          value: virtual-kubelet-cn-north-1b # vk node节点名称
        - name: VKUBELET_POD_IP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        volumeMounts:
        - name: configs
          mountPath: "/etc/virtual-kubelet/config"
          readOnly: true
        command: ["virtual-kubelet"]
        args: [
          "--provider", "jdcloud",
          "--nodename", "$(DEFAULT_NODE_NAME)",
          "--cluster-domain", "cluster.local",
          "--provider-config", "/etc/virtual-kubelet/config/nc.toml",
          "--os", "Linux",
          #"--metrics-addr", ":10255",
          "--no-verify-clients=true",
          #"--disable-taint=true",
          "--log-level", "info"
        ]
      volumes:
      - name: configs
        emptyDir: {}
      serviceAccountName: virtual-kubelet 
---
apiVersion: storage.k8s.io/v1
kind: CSINode
metadata:
  annotations:
    storage.alpha.kubernetes.io/migrated-plugins: kubernetes.io/aws-ebs,kubernetes.io/azure-disk,kubernetes.io/azure-file,kubernetes.io/cinder,kubernetes.io/gce-pd
  name: virtual-kubelet-cn-north-1b # vk node节点名称
spec:
  drivers:
    - allocatable:
        count: 100000
      name: zbs.csi.jdcloud.com
      nodeID: virtual-kubelet-cn-north-1b # vk node节点名称
      topologyKeys:
        - topology.zbs.csi.jdcloud.com/zone
    - name: nfs.csi.jdcloud.com
      nodeID: virtual-kubelet-cn-north-1b # vk node节点名称

部署验证

• 虚拟节点控制器pod 处于Running状态：kubectl get pod -n kube-system | grep virtual-kubelet；

• 虚拟节点处于Reday状态：kubectl get node | grep virtual-kubelet；

• CSINode成功创建：kubectl get csinodes.storage.k8s.io | grep virtual-kubelet ；