## 集群是分布式架构 1. 集群情况下每个节点之间会互相通信的; 2. 集群情况下每个节点都是可以读写的; 3. cluster 中Redis只有DB0没有DB1-15  ## Redis cluster架构 1. 节点 2. meet 3. 指派槽 4. 复制(每个主节点都有一个从节点) ## 节点 ``` cluster-enabled:yes //集群模式启动 ``` ## meet 每个节点和其他节点都是互相通信的;所有节点共享信息,这样每个节点才能知道哪个节点负责哪些槽位;  ## 指派槽 ## Redis cluster特性 1. 复制(每个主节点都有从节点) 2. 高可用 3. 分片 # Redis cluster安装配置 ## 原生命令安装-理解架构 1. 配置开启节点,最后一项配置是不合理的,正式项目要设置为no;  主要配置  2. meet ``` cluster meet ip port ``` 3. 指派槽 ``` cluster addslots slot [slot...] ``` 4. 主从关系 ``` cluster replicate node-id ``` ## 完整配置 将端口进行替换 ``` port 7001 dbfilename dump-7001.rdb cluster-enabled yes cluster-config-file nodes-7001.conf cluster-node-timeout 15000 ``` 查看进程 ``` ➜ ~ ps -ef | grep redis 501 9243 1376 0 10:47下午 ttys000 0:00.22 ./redis-server 127.0.0.1:7000 [cluster] 501 9356 2784 0 10:51下午 ttys001 0:00.04 ./redis-server 127.0.0.1:7001 [cluster] 501 9358 3064 0 10:51下午 ttys002 0:00.03 ./redis-server 127.0.0.1:7002 [cluster] 501 9360 3120 0 10:51下午 ttys003 0:00.02 ./redis-server 127.0.0.1:7003 [cluster] 501 9365 4897 0 10:51下午 ttys005 0:00.02 ./redis-server 127.0.0.1:7004 [cluster] 501 9368 5230 0 10:51下午 ttys006 0:00.02 ./redis-server 127.0.0.1:7005 [cluster] ``` 连接节点,输入命令发现不可用 ``` ➜ bin ./redis-cli -p 7000 127.0.0.1:7000> set name jack (error) CLUSTERDOWN Hash slot not served //不可用 ``` ``` ➜ bin ./redis-cli -p 7000 cluster nodes 8081564a18dc03a9d4384ed502fa40de22ca57fb :7000@17000 myself,master - 0 0 0 connected ``` 集群信息 ``` ➜ bin ./redis-cli -p 7000 cluster info cluster_state:fail //状态是失败的 cluster_slots_assigned:0 //分配的槽 cluster_slots_ok:0 //成功的槽 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:1 cluster_size:0 cluster_current_epoch:0 cluster_my_epoch:0 cluster_stats_messages_sent:0 cluster_stats_messages_received:0 ``` 集群配置文件,也就是cluster-config-file 配置的文件: ``` ➜ redis cat nodes-7000.conf 8081564a18dc03a9d4384ed502fa40de22ca57fb :0@0 myself,master - 0 0 0 connected vars currentEpoch 0 lastVoteEpoch 0 ``` * **集群模式的Redis除了原有的配置文件之外又加了一份集群配置文件**。 当集群内节点信息发生变化，如添加节点、节点下线、故障转移等。节点会自动保存集群状态到配置文件中。需要注意的是，**Redis自动维护集群配置文件，不要手动修改，**防止节点重启时产生集群信息错乱 * 6379节点启动成功，**第一次启动时如果没有集群配置文件，它会自动创建一份，**文件名称采用cluster-config-file参数项控制，建议采用node- {port}.conf格式定义，通过使用端口号区分不同节点，防止同一机器下多个 节点彼此覆盖，造成集群信息异常 * **如果启动时存在集群配置文件，节点会使用配置文件内容初始化集群信息**。启动过程如下图所示：  * 文件内容记录了集群初始状态，**这里最重要的是节点ID，它是一个40位16进制字符串，用于唯一标识集群内一个节点**，之后很多集群操作都要借助于节点ID来完成。**需要注意是，节点ID不同于运行ID：**节点ID在集群初始化 时只创建一次，节点重启时会加载集群配置文件进行重用，而Redis的运行 ID每次重启都会变化 * **在节点6380执行cluster nodes命令获取集群节点状态，**每个节点目前只能识别出自己的节点信息; ## meet 操作(握手) * 我们启动6个节点，但每个节点彼此并不知道对方的存在，**下面通过节点握手让6个节点彼此建立联系从而组成一个集群**; ``` ➜ bin ./redis-cli -p 7000 cluster meet 127.0.0.1 7001 //meet操作 OK ``` 执行cluster nodes可以看到握手成功 ``` ➜ bin ./redis-cli -p 7000 cluster nodes a4890ef360af0f5d9e772f5523ec46aed93af198 127.0.0.1:7001@17001 master - 0 1606057637751 0 connected 8081564a18dc03a9d4384ed502fa40de22ca57fb 127.0.0.1:7000@17000 myself,master - 0 0 1 connected ``` ``` ➜ bin ./redis-cli -p 7000 cluster meet 127.0.0.1 7002 OK ➜ bin ./redis-cli -p 7000 cluster meet 127.0.0.1 7003 OK ➜ bin ./redis-cli -p 7000 cluster meet 127.0.0.1 7004 OK ➜ bin ./redis-cli -p 7000 cluster meet 127.0.0.1 7005 OK ➜ bin ./redis-cli -p 7003 cluster nodes //可以看到该节点发现了其他节点,我们只需要在集群内任意节点上执行cluster meet命令加入新节点，握手状态会通过消息在集群内传播,这样其他节点会自动发现新节点并发起握手流程 f4956ec33caae7ab7ac00fa7bfac56d338f6cd87 127.0.0.1:7003@17003 myself,master - 0 1606057695000 0 connected e8a77578af06d00e95cffd7059313c979f208212 127.0.0.1:7004@17004 master - 0 1606057695000 5 connected a056926a23dc3a62c8da560c9ac6b4e29fd657cf 127.0.0.1:7005@17005 master - 0 1606057695000 4 connected 8081564a18dc03a9d4384ed502fa40de22ca57fb 127.0.0.1:7000@17000 master - 0 1606057696128 1 connected e1d86f41ed70cf14250d6727643a4828bc7acaf1 127.0.0.1:7002@17002 master - 0 1606057695104 3 connected a4890ef360af0f5d9e772f5523ec46aed93af198 127.0.0.1:7001@17001 master - 0 1606057697157 2 connected ``` ## 指派插槽 集群彼此发现,但是没有分配插槽; ``` ➜ bin ./redis-cli -p 7003 cluster nodes f4956ec33caae7ab7ac00fa7bfac56d338f6cd87 127.0.0.1:7003@17003 myself,master - 0 1606057695000 0 connected e8a77578af06d00e95cffd7059313c979f208212 127.0.0.1:7004@17004 master - 0 1606057695000 5 connected a056926a23dc3a62c8da560c9ac6b4e29fd657cf 127.0.0.1:7005@17005 master - 0 1606057695000 4 connected 8081564a18dc03a9d4384ed502fa40de22ca57fb 127.0.0.1:7000@17000 master - 0 1606057696128 1 connected e1d86f41ed70cf14250d6727643a4828bc7acaf1 127.0.0.1:7002@17002 master - 0 1606057695104 3 connected a4890ef360af0f5d9e772f5523ec46aed93af198 127.0.0.1:7001@17001 master - 0 1606057697157 2 connected ``` 指派插槽: ``` ➜ bin ./redis-cli -p 7000 cluster addslots {0..5461} OK ➜ bin ./redis-cli -p 7001 cluster addslots {5462..10922} OK ➜ bin ./redis-cli -p 7002 cluster addslots {10923..16383} OK ``` ``` ➜ bin ./redis-cli -p 7002 cluster nodes e8a77578af06d00e95cffd7059313c979f208212 127.0.0.1:7004@17004 slave a4890ef360af0f5d9e772f5523ec46aed93af198 0 1606059393911 5 connected 8081564a18dc03a9d4384ed502fa40de22ca57fb 127.0.0.1:7000@17000 master - 0 1606059393000 1 connected 0-5461 //0-5461就是分配的插槽 a4890ef360af0f5d9e772f5523ec46aed93af198 127.0.0.1:7001@17001 master - 0 1606059390875 2 connected 5462-10922 f4956ec33caae7ab7ac00fa7bfac56d338f6cd87 127.0.0.1:7003@17003 slave 8081564a18dc03a9d4384ed502fa40de22ca57fb 0 1606059391000 1 connected a056926a23dc3a62c8da560c9ac6b4e29fd657cf 127.0.0.1:7005@17005 slave e1d86f41ed70cf14250d6727643a4828bc7acaf1 0 1606059392894 4 connected e1d86f41ed70cf14250d6727643a4828bc7acaf1 127.0.0.1:7002@17002 myself,master - 0 1606059391000 3 connected 10923-16383 ``` 查看插槽: ``` ➜ bin ./redis-cli -p 7002 cluster slots 1) 1) (integer) 0 2) (integer) 5461 3) 1) "127.0.0.1" 2) (integer) 7000 3) "8081564a18dc03a9d4384ed502fa40de22ca57fb" 4) 1) "127.0.0.1" 2) (integer) 7003 3) "f4956ec33caae7ab7ac00fa7bfac56d338f6cd87" 2) 1) (integer) 5462 2) (integer) 10922 3) 1) "127.0.0.1" 2) (integer) 7001 3) "a4890ef360af0f5d9e772f5523ec46aed93af198" 4) 1) "127.0.0.1" 2) (integer) 7004 3) "e8a77578af06d00e95cffd7059313c979f208212" 3) 1) (integer) 10923 2) (integer) 16383 3) 1) "127.0.0.1" 2) (integer) 7002 3) "e1d86f41ed70cf14250d6727643a4828bc7acaf1" 4) 1) "127.0.0.1" 2) (integer) 7005 3) "a056926a23dc3a62c8da560c9ac6b4e29fd657cf" ``` ## 主从分配 我们的目的是7003对应7000,7004对应7001,7005对应7002; ``` ➜ bin ./redis-cli -p 7003 cluster replicate 8081564a18dc03a9d4384ed502fa40de22ca57fb //cluster replicate 主节点的node id OK ➜ bin ./redis-cli -p 7004 cluster replicate a4890ef360af0f5d9e772f5523ec46aed93af198 OK ➜ bin ./redis-cli -p 7005 cluster replicate e1d86f41ed70cf14250d6727643a4828bc7acaf1 OK ``` 查看主从关系: ``` ➜ bin ./redis-cli -p 7003 cluster nodes f4956ec33caae7ab7ac00fa7bfac56d338f6cd87 127.0.0.1:7003@17003 myself,slave 8081564a18dc03a9d4384ed502fa40de22ca57fb 0 1606058562000 0 connected //可以看到是8081564a18dc03a9d4384ed502fa40de22ca57fb的从节点 e8a77578af06d00e95cffd7059313c979f208212 127.0.0.1:7004@17004 slave a4890ef360af0f5d9e772f5523ec46aed93af198 0 1606058562031 5 connected a056926a23dc3a62c8da560c9ac6b4e29fd657cf 127.0.0.1:7005@17005 slave e1d86f41ed70cf14250d6727643a4828bc7acaf1 0 1606058562000 4 connected 8081564a18dc03a9d4384ed502fa40de22ca57fb 127.0.0.1:7000@17000 master - 0 1606058561017 1 connected e1d86f41ed70cf14250d6727643a4828bc7acaf1 127.0.0.1:7002@17002 master - 0 1606058563046 3 connected a4890ef360af0f5d9e772f5523ec46aed93af198 127.0.0.1:7001@17001 master - 0 1606058564072 2 connected ``` ## 使用集群 ``` ➜ bin ./redis-cli -c -p 7000 127.0.0.1:7000> set name jack -> Redirected to slot [5798] located at 127.0.0.1:7001 OK 127.0.0.1:7001> get name "jack" ``` ## 官方工具安装 1. 安装ruby; 2. 安装rubygem redis; 3. 安装redis-trib.rb; ## 使用redis-trib进行安装 * redis-cli --cluster命令本来是由redis-trib.rb工具提供的，但是随着发展，redis-trib.rb工具的功能被逐渐归纳到redis-cli工具中了,但是在Redis5.0版本以上才支持; 1. 所有的Redis.conf配合和手动的配置是一致的; 2. 将所有的Redis-server 启动; 3. 注意,节点下线需要删除配置或者启用新的端口(这里指的是cluster-config-file的配置文件). **输入下面的命令自动完成节点握手和槽分配。命令如下：** * \--cluster-replicas 1：指定集群中每个主节点配备几个从节点，这里设置为1。并且该命令会自己创建主节点和分配从节点，其中前3个是主节点，后3个是从节点，后3个从节点分别复制前3个主节点 ``` ./redis-cli --cluster create --cluster-replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 ``` * 输入上面的命令之后，中间会让你输入yes或no，下面我们输入yes开始执行节点握手和槽分配操作 ``` ➜ bin ./redis-cli --cluster create --cluster-replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 >>> Performing hash slots allocation on 6 nodes... Master[0] -> Slots 0 - 5460 Master[1] -> Slots 5461 - 10922 Master[2] -> Slots 10923 - 16383 Adding replica 127.0.0.1:7004 to 127.0.0.1:7000 Adding replica 127.0.0.1:7005 to 127.0.0.1:7001 Adding replica 127.0.0.1:7003 to 127.0.0.1:7002 >>> Trying to optimize slaves allocation for anti-affinity [WARNING] Some slaves are in the same host as their master M: e4bba455615996a2419f5ea68fb483572da69a4e 127.0.0.1:7000 slots:[0-5460] (5461 slots) master M: 26cdec411e03740e81741fd1af56c84a8995e1bf 127.0.0.1:7001 slots:[5461-10922] (5462 slots) master M: 86ebe5a560dc152174ae87b4555890486aa35051 127.0.0.1:7002 slots:[10923-16383] (5461 slots) master S: 6f0a7298bd81de901ebd4989504a4d914e5002e2 127.0.0.1:7003 replicates 86ebe5a560dc152174ae87b4555890486aa35051 S: f29d5a33719410d0d965f0cd04a1aaf62772113c 127.0.0.1:7004 replicates e4bba455615996a2419f5ea68fb483572da69a4e S: 4c0dcdf4862435f9e0b7ab9573c5d88ebce69c9e 127.0.0.1:7005 replicates 26cdec411e03740e81741fd1af56c84a8995e1bf Can I set the above configuration? (type 'yes' to accept): yes >>> Nodes configuration updated >>> Assign a different config epoch to each node >>> Sending CLUSTER MEET messages to join the cluster Waiting for the cluster to join . >>> Performing Cluster Check (using node 127.0.0.1:7000) M: e4bba455615996a2419f5ea68fb483572da69a4e 127.0.0.1:7000 slots:[0-5460] (5461 slots) master 1 additional replica(s) M: 26cdec411e03740e81741fd1af56c84a8995e1bf 127.0.0.1:7001 slots:[5461-10922] (5462 slots) master 1 additional replica(s) S: 6f0a7298bd81de901ebd4989504a4d914e5002e2 127.0.0.1:7003 slots: (0 slots) slave replicates 86ebe5a560dc152174ae87b4555890486aa35051 S: 4c0dcdf4862435f9e0b7ab9573c5d88ebce69c9e 127.0.0.1:7005 slots: (0 slots) slave replicates 26cdec411e03740e81741fd1af56c84a8995e1bf M: 86ebe5a560dc152174ae87b4555890486aa35051 127.0.0.1:7002 slots:[10923-16383] (5461 slots) master 1 additional replica(s) S: f29d5a33719410d0d965f0cd04a1aaf62772113c 127.0.0.1:7004 slots: (0 slots) slave replicates e4bba455615996a2419f5ea68fb483572da69a4e [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered. ```  * 最后的输出报告说明：16384个槽全部被分配，集群创建成功。这里需要注意命令中节点的地址必须是不包含任何槽/数据的节点，否则会拒绝创建集群 * **备注：**如果只想创建主节点，而不同时创建从节点，那么需要忽略--cluster-replicas 1参数。命令如下，例如下面只创建3个集群主节点; ``` redis-cli --cluster create 127.0.0.1:6379 127.0.0.1:6380 127.0.0.1:638 ``` ## 集群完整性检查 * 集群完整性指所有的槽都分配到存活的主节点上，只要16384个槽中有 一个没有分配给节点则表示集群不完整 * 可以使用下面的命令检测之前创建的两个集群是否成功，check命令只需要给出集群中任意一个节点地址就可以完成整个集群的检查工作： ``` ./redis-cli --cluster check 127.0.0.1:7000 ``` 检查结果 ``` ➜ bin ./redis-cli --cluster check 127.0.0.1:7000 127.0.0.1:7000 (e4bba455...) -> 0 keys | 5461 slots | 1 slaves. 127.0.0.1:7001 (26cdec41...) -> 0 keys | 5462 slots | 1 slaves. 127.0.0.1:7002 (86ebe5a5...) -> 0 keys | 5461 slots | 1 slaves. [OK] 0 keys in 3 masters. 0.00 keys per slot on average. >>> Performing Cluster Check (using node 127.0.0.1:7000) M: e4bba455615996a2419f5ea68fb483572da69a4e 127.0.0.1:7000 slots:[0-5460] (5461 slots) master 1 additional replica(s) M: 26cdec411e03740e81741fd1af56c84a8995e1bf 127.0.0.1:7001 slots:[5461-10922] (5462 slots) master 1 additional replica(s) S: 6f0a7298bd81de901ebd4989504a4d914e5002e2 127.0.0.1:7003 slots: (0 slots) slave replicates 86ebe5a560dc152174ae87b4555890486aa35051 S: 4c0dcdf4862435f9e0b7ab9573c5d88ebce69c9e 127.0.0.1:7005 slots: (0 slots) slave replicates 26cdec411e03740e81741fd1af56c84a8995e1bf M: 86ebe5a560dc152174ae87b4555890486aa35051 127.0.0.1:7002 slots:[10923-16383] (5461 slots) master 1 additional replica(s) S: f29d5a33719410d0d965f0cd04a1aaf62772113c 127.0.0.1:7004 slots: (0 slots) slave replicates e4bba455615996a2419f5ea68fb483572da69a4e [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered. ``` ## 总结 1. 原生命令安装: 1. 理解Redis cluster; 2. 生产环境不使用(繁琐,产生错误的概率较大); 2. 官方工具安装 1. 高效,准确; 3. 其他 1.可视化部署