centos7部署etcd节点 · 运维心得

简介：Etcd 是什么 * etcd是一个分布式可靠的键值存储，用于分布式系统的最关键数据，重点是： 1.*简单*：定义明确，面向用户的API（gRPC） 2.*安全*：具有可选客户端证书身份验证的自动TLS 3.*快速*：基准测试10,000次/秒 4.*可靠*：使用Raft一致性算法分布集群 * etcd是用Go编写的，使用[Raft](https://raft.github.io/)一致性算法来管理高度可用的复制日志。 * etcd被[许多公司](https://github.com/coreos/etcd/blob/master/Documentation/production-users.md)用于[生产](https://github.com/coreos/etcd/blob/master/Documentation/production-users.md)，开发团队在关键部署场景中支持它，其中etcd经常与[Kubernetes](http://kubernetes.io/)，[locksmith](https://github.com/coreos/locksmith)，[vulcand](https://github.com/vulcand/vulcand)，[Doorman](https://github.com/youtube/doorman)等许多应用程序配合使用。通过**严格的测试**进一步确保可靠性。 * 有关简单的命令行客户端，请参阅[etcdctl](https://github.com/coreos/etcd/tree/master/etcdctl)(etcdctl --helper) ### 一、ETCD单机部署 ##### 1、下载二进制包 https://github.com/coreos/etcd/releases ``` wget https://github.com/etcd-io/etcd/releases/download/v3.3.12/etcd-v3.3.12-linux-arm64.tar.gz ``` ##### 2、解压缩 ``` tar -zxvf etcd-v3.3.12-linux-arm64.tar.gz ``` ##### 3、设置环境变量 ``` export ETCDPATH="/home/etcd/etcd-v3.3.12-linux-amd64" export ETCDCTL_API=3 export PATH="$PATH:$ETCDPATH" ``` ##### 4、启动etcd ``` etcd --data-dir $ETCDPATH/test_data ``` ##### 5、客户端测试 ``` etcdctl --endpoints=http://127.0.0.1:2379 put foo bar etcdctl --endpoints=http://127.0.0.1:2379 get foo ``` ``` [root@bogon /]# etcdctl --endpoints=http://127.0.0.1:2379 put foo bar OK [root@bogon /]# etcdctl --endpoints=http://127.0.0.1:2379 get foo foo bar ``` ### 二、docker单机部署ETCD ##### 1、拉取etcd镜像 ``` docker pull quay.io/coreos/etcd:v3.3.9 ``` ##### 2、设置环境变量和监听地址 ``` -e ETCDCTL_API=3 -p 2379:2379 -p 2380:2380 ``` ##### 3、运行etcd容器 ``` docker run -d -it --rm \ --name etcd_test \ -e ETCDCTL_API=3 \ -p 2379:2379 \ -p 2380:2380 \ quay.io/coreos/etcd:v3.3.9 \ etcd \ --advertise-client-urls http://0.0.0.0:2379 \ --listen-client-urls http://0.0.0.0:2379 ``` 容器退出自动删除rm ``` [root@bogon etcd]# docker run -d -it --rm --name etcd_test -e ETCDCTL_API=3 -p 2379:2379 -p 2380:2380 quay.io/coreos/etcd:v3.3.9 etcd --advertise-client-urls http://0.0.0.0:2379 --listen-client-urls http://0.0.0.0:2379 ca444c094b6bf00b47726b0dee600620d3962cd0fb78d224db9fae12da94dc13 ``` 容器停止总是重启 ``` [root@ansible-server ~]# docker run -d -it --restart=always --name etcd1 -e ETCDCTL_API=3 -p 2379:2379 -p 2380:2380 quay.io/coreos/etcd:v3.3.9 etcd --advertise-client-urls http://0.0.0.0:2379 --listen-client-urls http://0.0.0.0:2379 199b521dcdd1a742701350c8024dd76c179db2ded0d05a40dd44adfd14f2fb27 ``` 其他状态： * no，默认策略，在容器退出时不重启容器 * on-failure，在容器非正常退出时（退出状态非0），才会重启容器 * on-failure:3，在容器非正常退出时重启容器，最多重启3次 * always，在容器退出时总是重启容器 * unless-stopped，在容器退出时总是重启容器，但是不考虑在Docker守护进程启动时就已经停止了的容器 ``` [root@bogon etcd]# docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES ca444c094b6b quay.io/coreos/etcd:v3.3.9 "etcd --advertise-..." 4 seconds ago Up 3 seconds 0.0.0.0:2379-2380->2379-2380/tcp etcd_test ``` ##### 4、客户端测试 ``` etcdctl --endpoints=http://127.0.0.1:2379 put foo bar etcdctl --endpoints=http://127.0.0.1:2379 get foo ``` ``` [root@bogon etcd]# etcdctl --endpoints=http://127.0.0.1:2379 put foo bar OK [root@bogon etcd]# etcdctl --endpoints=http://127.0.0.1:2379 get foo foo bar [root@bogon etcd]# etcdctl --endpoints=http://127.0.0.1:2379 put foo bar1 OK [root@bogon etcd]# etcdctl --endpoints=http://127.0.0.1:2379 get foo foo bar1 ``` ### 三、Etcd本地集群（使用goreman管理） #### 1、下载etcd二进制包&&设置环境变量&&参考Etcd单机部署 #### 2、安装goreman，安装goreman前需要安装go环境和git。 #### 3、安装go环境 ``` wget https://studygolang.com/dl/golang/go1.10.1.linux-amd64.tar.gz tar -xvf go1.10.1.linux-amd64.tar.gz vim /etc/profile export GOROOT=/home/go/go export GOPATH=/home/go/data export PATH=$PATH:$GOROOT/bin:$GOPATH/bin source /etc/profile ``` #### 4、安装git ``` yum -y install git ``` #### 5、安装 goreman ``` go get github.com/mattn/goreman ``` 下载后的文件在```export GOPATH=/home/go/data```设置的路径中 #### 6、编写Procfile（管理集群） [https://github.com/coreos/etcd/blob/master/Procfile](https://github.com/coreos/etcd/blob/master/Procfile) #### 7、启动etcd（goreman runlist） ``` goreman -f Procfile start ``` ### 四、docker cluster（集群） #### 8、etcd集群 ``` docker pull busybox ``` ##### docker netword ``` #docker network create etcd_cluster docker network create --subnet 172.16.3.0/16 etcd_cluster docker network inspect etcd_cluster #docker network rm etcd_cluster docker run -itd --rm --network etcd_cluster --ip 172.16.3.0 --name test busybox ``` ``` docker network ls docker network rm etcd_cluster ``` #### 9、etcd cluster ``` # etcd1 docker run -itd --restart=always \ --network etcd_cluster \ --ip 172.16.3.31 \ --hostname etcd1 \ --name etcd1 \ -e ETCDCTL_API=3 \ -p 12379:2379 \ -p 12380:2380 \ quay.io/coreos/etcd:v3.3.9 \ etcd --name etcd1 \ --initial-advertise-peer-urls http://172.16.3.31:2380 \ --listen-peer-urls http://172.16.3.31:2380 \ --listen-client-urls http://172.16.3.31:2379,http://127.0.0.1:2379 \ --advertise-client-urls http://172.16.3.31:2379 \ --initial-cluster-token etcd-cluster-1 \ --initial-cluster etcd1=http://172.16.3.31:2380,etcd2=http://172.16.3.32:2380,etcd3=http://172.16.3.33:2380 \ --initial-cluster-state new # etcd2 docker run -itd --restart=always \ --network etcd_cluster \ --ip 172.16.3.32 \ --hostname etcd2 \ --name etcd2 \ -e ETCDCTL_API=3 \ -p 22379:2379 \ -p 22380:2380 \ quay.io/coreos/etcd:v3.3.9 \ etcd --name etcd2 \ --initial-advertise-peer-urls http://172.16.3.32:2380 \ --listen-peer-urls http://172.16.3.32:2380 \ --listen-client-urls http://172.16.3.32:2379,http://127.0.0.1:2379 \ --advertise-client-urls http://172.16.3.32:2379 \ --initial-cluster-token etcd-cluster-1 \ --initial-cluster etcd1=http://172.16.3.31:2380,etcd2=http://172.16.3.32:2380,etcd3=http://172.16.3.33:2380 \ --initial-cluster-state new # etcd3 docker run -itd --restart=always \ --network etcd_cluster \ --ip 172.16.3.33 \ --hostname etcd3 \ --name etcd3 \ -e ETCDCTL_API=3 \ -p 32379:2379 \ -p 32380:2380 \ quay.io/coreos/etcd:v3.3.9 \ etcd --name etcd3 \ --initial-advertise-peer-urls http://172.16.3.33:2380 \ --listen-peer-urls http://172.16.3.33:2380 \ --listen-client-urls http://172.16.3.33:2379,http://127.0.0.1:2379 \ --advertise-client-urls http://172.16.3.33:2379 \ --initial-cluster-token etcd-cluster-1 \ --initial-cluster etcd1=http://172.16.3.31:2380,etcd2=http://172.16.3.32:2380,etcd3=http://172.16.3.33:2380 \ --initial-cluster-state new # client #进入容器 docker exec -it etcd1 bin/sh #客户端测试 etcdctl --write-out=table --endpoints=http://127.0.0.1:2379 member list etcdctl --endpoints=http://127.0.0.1:2379 put foo bar etcdctl --endpoints=http://127.0.0.1:2379 get foo #测试结果如下 [root@bogon ~]# docker exec -it etcd1 bin/sh / # etcdctl --write-out=table --endpoints=http://127.0.0.1:2379 member list +------------------+---------+-------+-------------------------+-------------------------+ | ID | STATUS | NAME | PEER ADDRS | CLIENT ADDRS | +------------------+---------+-------+-------------------------+-------------------------+ | c26d6ba798c079c | started | etcd3 | http://172.16.3.33:2380 | http://172.16.3.33:2379 | | 4631df2115e1ef72 | started | etcd2 | http://172.16.3.32:2380 | http://172.16.3.32:2379 | | a92fe5422902bc40 | started | etcd1 | http://172.16.3.31:2380 | http://172.16.3.31:2379 | +------------------+---------+-------+-------------------------+-------------------------+ / # etcdctl --endpoints=http://127.0.0.1:2379 put foo bar OK / # etcdctl --endpoints=http://127.0.0.1:2379 get foo foo bar #docker ps [root@bogon docker]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 5988c4443eb5 quay.io/coreos/etcd:v3.3.9 "etcd --name etcd3..." 37 seconds ago Up 36 seconds 0.0.0.0:32379->2379/tcp, 0.0.0.0:32380->2380/tcp etcd3 d4fdda6400cc quay.io/coreos/etcd:v3.3.9 "etcd --name etcd2..." 44 seconds ago Up 44 seconds 0.0.0.0:22379->2379/tcp, 0.0.0.0:22380->2380/tcp etcd2 af86e5fd2ae5 quay.io/coreos/etcd:v3.3.9 "etcd --name etcd1..." 53 seconds ago Up 52 seconds 0.0.0.0:2379-2380->2379-2380/tcp etcd1 ```