# 华为云cce 部署 多节点nacos ## 在k8s中部署的难点 在k8s中部署nacos集群和在裸机器上直接部署nacos机器其实差别不大。 最主要的区别是k8s中部署的服务没有固定的ip地址，而nacos集群部署需要配置所有实例的ip ## 2 解决 * 在k8s中通过StatefulSet和Headless Service为每个nacos实例生成一个唯一的dns地址， * 创建一个普通Service给可客户端使用 ## 3 实现方式 ### 3.1 创建数据库配置 ———————————————— ``` --- apiVersion: v1 kind: Secret metadata: name: nacos type: Opaque data: mysql.db.host: "NDFjZTgyZjY4OWE5NGU4ZDk4YmRlOWQ2MDQ2ZTA0YWRpbjAxLmludGVybmFsLmNuLWVhc3QtMy5teXNxbC5yZHMubXlodWF3ZWljbG91ZC5jb20=" mysql.db.name: "bmFjb3M=" mysql.db.port: "MzMwNg==" mysql.db.user: "bmFjb3M=" mysql.db.password: "V0ExNmdvVWE2bU5oUmlqRg==" --- ``` > Opaque Secret Opaque 类型的数据是一个 map 类型，要求value是base64编码格式。 比如我们来创建一个用户名为 nacos, 的 Secret 对象， 首先我们先把这用户名和密码做 base64 编码 ``` $ echo -n 'nacos' | openssl base64 bmFjb3M= ``` ### 3.2 部署Headless Service ———————————————— Headless Service为每个pod（nacos实例）生成一个DNS地址，用作NACOS\_SERVERS配置 ``` --- apiVersion: v1 kind: Service metadata: name: nacos-headless labels: app: nacos annotations: service.alpha.kubernetes.io/tolerate-unready-endpoints: "true" spec: ports: - port: 8848 name: server targetPort: 8848 - port: 9848 name: client-rpc targetPort: 9848 - port: 9849 name: raft-rpc targetPort: 9849 ## 兼容1.4.x版本的选举端口 - port: 7848 name: old-raft-rpc targetPort: 7848 clusterIP: None selector: app: nacos --- ``` ### 3.3 通过StatefulSet部署nacos StatefulSet部署方式为每个POD生成固定的名称，如nacos-0、nacos-1、nacos-2等。 ``` apiVersion: apps/v1 kind: StatefulSet metadata: name: nacos spec: serviceName: nacos-headless replicas: 3 template: metadata: labels: app: nacos annotations: pod.alpha.kubernetes.io/initialized: "true" spec: volumes: - name: vol-163912341665228473 hostPath: path: /opt/logs/ type: '' affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchExpressions: - key: "app" operator: In values: - nacos topologyKey: "kubernetes.io/hostname" initContainers: - name: peer-finder-plugin-install image: nacos/nacos-peer-finder-plugin:1.1 imagePullPolicy: IfNotPresent volumeMounts: - mountPath: /home/nacos/plugins/peer-finder name: data subPath: peer-finder containers: - name: nacos imagePullPolicy: Always image: swr.cn-east-3.myhuaweicloud.com/huyi-base/nacos-server:2.0.3 resources: limits: cpu: '2' memory: 4Gi requests: memory: '4Gi' cpu: '2' ports: - containerPort: 8848 name: client-port - containerPort: 9848 name: client-rpc - containerPort: 9849 name: raft-rpc - containerPort: 7848 name: old-raft-rpc env: - name: NACOS_REPLICAS value: "3" - name: SERVICE_NAME value: "nacos-headless" - name: DOMAIN_NAME value: "cluster.local" - name: POD_NAMESPACE valueFrom: fieldRef: apiVersion: v1 fieldPath: metadata.namespace - name: MYSQL_SERVICE_HOST valueFrom: secretKeyRef: name: nacos key: mysql.db.host - name: MYSQL_SERVICE_DB_NAME valueFrom: secretKeyRef: name: nacos key: mysql.db.name - name: MYSQL_SERVICE_PORT valueFrom: secretKeyRef: name: nacos key: mysql.db.port - name: MYSQL_SERVICE_USER valueFrom: secretKeyRef: name: nacos key: mysql.db.user - name: MYSQL_SERVICE_PASSWORD valueFrom: secretKeyRef: name: nacos key: mysql.db.password - name: NACOS_SERVER_PORT value: "8848" - name: NACOS_APPLICATION_PORT value: "8848" - name: PREFER_HOST_MODE value: "hostname" volumeMounts: - name: data mountPath: /home/nacos/plugins/peer-finder subPath: peer-finder - name: data mountPath: /home/nacos/data subPath: data - name: vol-163912341665228473 mountPath: /home/nacos/logs/ policy: logs: rotate: Hourly annotations: format: '{"multi":{"mode":"time","value":"YYYY-MM-DD hh:mm:ss"}}' pathPattern: nacos.log imagePullSecrets: - name: default-secret volumeClaimTemplates: - metadata: name: data annotations: everest.io/disk-volume-type: SAS labels: failure-domain.beta.kubernetes.io/region: cn-east-3 failure-domain.beta.kubernetes.io/zone: cn-east-3a spec: accessModes: [ "ReadWriteOnce" ] resources: requests: storage: 20Gi storageClassName: csi-disk selector: matchLabels: app: nacos ``` ### 初始化建表 mysql [MySQL 建表](https://github.com/alibaba/nacos/blob/master/distribution/conf/nacos-mysql.sql)