一、自签APIServer SSL证书 1、 进入TLS目录 1.1、执行shell： ``` generate_k8s_cert.sh ``` shell命令的内容是： ``` cfssl gencert -initca ca-csr.json | cfssljson -bare ca - cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes server-csr.json | cfssljson -bare server cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes kube-proxy-csr.json | cfssljson -bare kube-proxy ``` 1.2、 server-csr.json node节点不用配置 ``` {     "CN": "kubernetes",     "hosts": [       "10.0.0.1",       "127.0.0.1",       "kubernetes",       "kubernetes.default",       "kubernetes.default.svc",       "kubernetes.default.svc.cluster",       "kubernetes.default.svc.cluster.local",       "192.168.72.166",       "192.168.72.167",       "192.168.72.170",       "192.168.72.171",       "192.168.72.172"     ],     "key": {         "algo": "rsa",         "size": 2048     },     "names": [         {             "C": "CN",             "L": "BeiJing",             "ST": "BeiJing",             "O": "k8s",             "OU": "System"         }     ] } ``` 1.3、 kube-proxy-csr.json ``` {   "CN": "system:kube-proxy",   "hosts": [],   "key": {     "algo": "rsa",     "size": 2048   },   "names": [     {       "C": "CN",       "L": "BeiJing",       "ST": "BeiJing",       "O": "k8s",       "OU": "System"     }   ] } ``` 2、生产证书 ``` ./generate_k8s_cert.sh ``` 二、部署Master组件 1、下载 https://github.com/kubernetes/kubernetes/releases/tag/v1.17.4 https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.17.md#server-binaries https://kubernetes.io/docs/reference/command-line-tools-reference/kube-proxy/ 2、Master组件 2.1、 kube-apiserver 1） kube-apiserver.service ``` [Unit] Description=Kubernetes API Server Documentation=https://github.com/kubernetes/kubernetes [Service] EnvironmentFile=/opt/kubernetes/cfg/kube-apiserver.conf ExecStart=/opt/kubernetes/bin/kube-apiserver $KUBE_APISERVER_OPTS Restart=on-failure [Install] WantedBy=multi-user.target ``` 2）kube-apiserver.conf ``` KUBE_APISERVER_OPTS="--logtostderr=false \ --v=2 \ --log-dir=/opt/kubernetes/logs \ --etcd-servers=https://192.168.254.201:2379,https://192.168.254.203:2379,https://192.168.254.204:2379 \ --bind-address=192.168.254.201 \ #监听的地址 --secure-port=6443 \ --advertise-address=192.168.254.201 \ #通告的地址 --allow-privileged=true \ #所有权限 --service-cluster-ip-range=10.0.0.0/24 \ #service的ip范围 --service-node-port-range=1-65535 \ --enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota,NodeRestriction \ #准入的插件 --authorization-mode=RBAC,Node \ #授权的模式 --enable-bootstrap-token-auth=true \ #自动颁发证书 --token-auth-file=/opt/kubernetes/cfg/token.csv \ --service-node-port-range=30000-32767 \ #暴露的端口范围 --kubelet-client-certificate=/opt/kubernetes/ssl/server.pem \ --kubelet-client-key=/opt/kubernetes/ssl/server-key.pem \ --tls-cert-file=/opt/kubernetes/ssl/server.pem  \ --tls-private-key-file=/opt/kubernetes/ssl/server-key.pem \ --client-ca-file=/opt/kubernetes/ssl/ca.pem \ --service-account-key-file=/opt/kubernetes/ssl/ca-key.pem \ --etcd-cafile=/opt/etcd/ssl/ca.pem \ --etcd-certfile=/opt/etcd/ssl/server.pem \ --etcd-keyfile=/opt/etcd/ssl/server-key.pem \ --audit-log-maxage=30 \ --audit-log-maxbackup=3 \ --audit-log-maxsize=100 \ --audit-log-path=/opt/kubernetes/logs/k8s-audit.log" ``` 2.2、kube-controller-manager 1）kube-controller-manager ``` [Unit] Description=Kubernetes Controller Manager Documentation=https://github.com/kubernetes/kubernetes [Service] EnvironmentFile=/opt/kubernetes/cfg/kube-controller-manager.conf ExecStart=/opt/kubernetes/bin/kube-controller-manager $KUBE_CONTROLLER_MANAGER_OPTS Restart=on-failure [Install] WantedBy=multi-user.target ``` 2）kube-controller-manager.conf ``` KUBE_CONTROLLER_MANAGER_OPTS="--logtostderr=false \ --v=2 \ --log-dir=/opt/kubernetes/logs \ --leader-elect=true \ #集群的选举 --master=127.0.0.1:8080 \ --address=127.0.0.1 \ --allocate-node-cidrs=true \ --cluster-cidr=10.244.0.0/16 \ --service-cluster-ip-range=10.0.0.0/24 \ --cluster-signing-cert-file=/opt/kubernetes/ssl/ca.pem \ --cluster-signing-key-file=/opt/kubernetes/ssl/ca-key.pem  \ --root-ca-file=/opt/kubernetes/ssl/ca.pem \ --service-account-private-key-file=/opt/kubernetes/ssl/ca-key.pem \ --experimental-cluster-signing-duration=87600h0m0s" ``` 2.3、 kube-scheduler 1）kube-scheduler.service ``` [Unit] Description=Kubernetes Scheduler Documentation=https://github.com/kubernetes/kubernetes [Service] EnvironmentFile=/opt/kubernetes/cfg/kube-scheduler.conf ExecStart=/opt/kubernetes/bin/kube-scheduler $KUBE_SCHEDULER_OPTS Restart=on-failure [Install] WantedBy=multi-user.target ``` 2）kube-scheduler.conf ``` KUBE_SCHEDULER_OPTS="--logtostderr=false \ --v=2 \ --log-dir=/opt/kubernetes/logs \ --leader-elect \ --master=127.0.0.1:8080 \ --address=127.0.0.1" ``` 3、实验 3.1、上传tar并且解压 3.2、拷贝证书到ssl里面 我们拷贝生成的证书到kubernetes中的ssl文件中 ``` cp /yhj/TLS/k8s/*.pem /yhj/kubernetes/ssl/ ```  3.3、cp -rf kubernetes /opt  3.4、拷贝service到执行目录下 ``` cp kube-apiserver.service kube-controller-manager.service kube-scheduler.service /usr/lib/systemd/system ```  3.5、启动 ``` # systemctl start kube-apiserver # systemctl start kube-controller-manager # systemctl start kube-scheduler # systemctl enable kube-apiserver # systemctl enable kube-controller-manager # systemctl enable kube-scheduler #systemctl restart kube-apiserver for i in $(ls /opt/kubernetes/bin);do systemctl enable $i;done ``` 3.6、日志查看 ``` less /opt/kubernetes/logs ``` ``` cd /opt/kubernetes/logs ```  ``` less kube-apiserver.INFO ``` 3.7、开机启动 subtopic 3.8、kubectl get node 3.9、查看 ``` cd /opt/kubernetes/bin/ ```  ``` ./kubectl get cs ```  3.10、验证 ``` ps -ef | grep kube ps -ef | grep kube | wc -4 ``` 4、 授权启用TLS Bootstrapping 4.1、文件拷贝  4.2、cat /opt/kubernetes/cfg/token.csv 格式：token,用户,uid,用户组 c47ffb939f5ca36231d9e3121a252940,kubelet-bootstrap,10001,"system:node-bootstrapper" 4.3、给kubelet-bootstrap授权： ``` kubectl create clusterrolebinding kubelet-bootstrap --clusterrole=system:node-bootstrapper --user=kubelet-bootstrap ``` 4.4、token也可自行生成替换： ``` head -c 16 /dev/urandom | od -An -t x | tr -d ' ' ``` 注意：我们如果需要最新的，我们只需要下载最新版，然后替换掉bin目录中的文件即可