redisCluster集群搭建与扩容

RedisCluster最少需要三台主服务器，三台从服务器。

1. 集群规划：

2. redis安装

CentOS 7 作为安装环境。

第一步：安装 C 语言需要的 GCC 环境

yum install -y gcc-c++

第二步：下载并解压缩 Redis 源码压缩包

wget http://download.redis.io/releases/redis-5.0.5.tar.gz tar -zvxf redis-5.0.5.tar.gz

第三步：编译 Redis 源码，进入 redis-5.0.5 目录，执行编译命令

cd redis-5.0.5/src make

第四步：安装 Redis ，需要通过 PREFIX 指定安装路径

mkdir /usr/redis -p cd redis-5.0.5/ #(需要到这个目录 否则报错:make: *** No rule to make target `install'. Stop. ) make install PREFIX=/usr/redis

安装完成后，指定目录/usr/redis目录下出现bin目录。

[root@cluster51:/opt/app]$ cd /usr/redis/ [root@cluster51:/usr/redis]$ ll total 0 drwxr-xr-x 2 root root 134 Aug 24 23:05 bin [root@cluster51:/usr/redis]$ cd bin/ [root@cluster51:/usr/redis/bin]$ ll total 32736 -rwxr-xr-x 1 root root 4366624 Aug 24 23:05 redis-benchmark -rwxr-xr-x 1 root root 8111832 Aug 24 23:05 redis-check-aof -rwxr-xr-x 1 root root 8111832 Aug 24 23:05 redis-check-rdb -rwxr-xr-x 1 root root 4806848 Aug 24 23:05 redis-cli lrwxrwxrwx 1 root root 12 Aug 24 23:05 redis-sentinel -> redis-server -rwxr-xr-x 1 root root 8111832 Aug 24 23:05 redis-server [root@cluster51:/usr/redis/bin]$

3. 集群安装

mkdir /opt/app/redis-cluster/6379 -p cd /opt/app/redis-5.0.5 make install PREFIX=/opt/app/redis-cluster/6379

创建6379实例，拷贝redis.conf文件到bin目录。

cp /opt/app/redis-5.0.5/redis.conf /opt/app/redis-cluster/6379/bin/

编辑redis.conf文件，注销掉bind ，修改port为6379。

66 # IF YOU ARE SURE YOU WANT YOUR INSTANCE TO LISTEN TO ALL THE INTERFACES 67 # JUST COMMENT THE FOLLOWING LINE. 68 # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 69 #bind 127.0.0.1 ​ 84 # By default protected mode is enabled. You should disable it only if 85 # you are sure you want clients from other hosts to connect to Redis 86 # even if no authentication is configured, nor a specific set of interfaces 87 # are explicitly listed using the "bind" directive. 88 protected-mode no 90 # Accept connections on the specified port, default is 6379 (IANA #815344). 91 # If port 0 is specified Redis will not listen on a TCP socket. 92 port 6379

修改redis.conf配置文件，daemonize 改为yes，打开cluster-enable yes.

132 ################################# GENERAL ##################################### 133 134 # By default Redis does not run as a daemon. Use 'yes' if you need it. 135 # Note that Redis will write a pid file in /var/run/redis.pid when daemonized. 136 daemonize yes 826 ################################ REDIS CLUSTER ############################### 827 828 # Normal Redis instances can't be part of a Redis Cluster; only nodes that are 829 # started as cluster nodes can. In order to start a Redis instance as a 830 # cluster node enable the cluster support uncommenting the following: 831 # 832 cluster-enabled yes

复制6379，创建其他实例，注意host和端口修改。

cd /opt/app/redis-cluster cp -r 6379/ 6380

拷贝到其他两台机器的对应目录下。

在cluster119和cluster137创建redis-cluster目录,并执行scp命令

mkdir -p /opt/app/redis-cluster scp -r 6379/ root@cluster119:/opt/app/redis-cluster/6379 scp -r 6379/ root@cluster137:/opt/app/redis-cluster/6379 scp -r 6380/ root@cluster137:/opt/app/redis-cluster/6380 scp -r 6380/ root@cluster119:/opt/app/redis-cluster/6380

创建start.sh，启动所有的实例。

cd 6379/bin ./redis-server redis.conf cd .. cd .. cd 6380/bin ./redis-server redis.conf

chmod u+x start.sh (赋写和执行的权限)

创建Redis集群（创建时Redis里不要有数据）

进入任意一台节点,执行以下命令，创建集群（指定ip:port，要指定ip不要使用hostname，否则报错）(前面三个为master后面三个实例为slave)

[root@cluster119 bin]# ./redis-cli --cluster create 172.17.18.116:6379 172.17.18.164:6379 172.17.19.51:6379 172.17.18.164:6380 172.17.18.116:6380 172.17.19.51:6380 --cluster-replicas 1 >>> Performing hash slots allocation on 6 nodes... Master[0] -> Slots 0 - 5460 Master[1] -> Slots 5461 - 10922 Master[2] -> Slots 10923 - 16383 Adding replica 172.17.18.164:6380 to 172.17.18.116:6379 Adding replica 172.17.19.51:6380 to 172.17.18.164:6379 Adding replica 172.17.18.116:6380 to 172.17.19.51:6379 M: c246ddcbdf31625a21d24815952a6eb39cfb4dde 172.17.18.116:6379 slots:[0-5460] (5461 slots) master M: e68cc0fcc2f27cf289e0606ca230864a3bddea0a 172.17.18.164:6379 slots:[5461-10922] (5462 slots) master M: 980182b3a13d4617fe9d757eb5c894ab2f2a070b 172.17.19.51:6379 slots:[10923-16383] (5461 slots) master S: e031c734f1612f6b62389c362133e646946d3628 172.17.18.164:6380 replicates c246ddcbdf31625a21d24815952a6eb39cfb4dde S: cbfef021f44ead084cf61fad4d96814bbed19300 172.17.18.116:6380 replicates 980182b3a13d4617fe9d757eb5c894ab2f2a070b S: fa3d71f57cde8605bc41836168312a443cf8216f 172.17.19.51:6380 replicates e68cc0fcc2f27cf289e0606ca230864a3bddea0a Can I set the above configuration? (type 'yes' to accept): yes >>> Nodes configuration updated >>> Assign a different config epoch to each node >>> Sending CLUSTER MEET messages to join the cluster Waiting for the cluster to join ...... >>> Performing Cluster Check (using node 172.17.18.116:6379) M: c246ddcbdf31625a21d24815952a6eb39cfb4dde 172.17.18.116:6379 slots:[0-5460] (5461 slots) master 1 additional replica(s) S: cbfef021f44ead084cf61fad4d96814bbed19300 172.17.18.116:6380 slots: (0 slots) slave replicates 980182b3a13d4617fe9d757eb5c894ab2f2a070b M: 980182b3a13d4617fe9d757eb5c894ab2f2a070b 172.17.19.51:6379 slots:[10923-16383] (5461 slots) master 1 additional replica(s) M: e68cc0fcc2f27cf289e0606ca230864a3bddea0a 172.17.18.164:6379 slots:[5461-10922] (5462 slots) master 1 additional replica(s) S: e031c734f1612f6b62389c362133e646946d3628 172.17.18.164:6380 slots: (0 slots) slave replicates c246ddcbdf31625a21d24815952a6eb39cfb4dde S: fa3d71f57cde8605bc41836168312a443cf8216f 172.17.19.51:6380 slots: (0 slots) slave replicates e68cc0fcc2f27cf289e0606ca230864a3bddea0a [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered.

命令客户端连接集群

#-c 表示是以redis集群方式进行连接 ./redis-cli -h 127.0.0.1 -p 7001 -c

查看集群状态

127.0.0.1:6379> cluster info cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:6 cluster_my_epoch:1 cluster_stats_messages_ping_sent:1350 cluster_stats_messages_pong_sent:1443 cluster_stats_messages_sent:2793 cluster_stats_messages_ping_received:1438 cluster_stats_messages_pong_received:1350 cluster_stats_messages_meet_received:5 cluster_stats_messages_received:2793

查看集群中的节点

127.0.0.1:6379> cluster nodes c246ddcbdf31625a21d24815952a6eb39cfb4dde 172.17.18.116:6379@16379 myself,master - 0 1598327174000 1 connected 0-5460 cbfef021f44ead084cf61fad4d96814bbed19300 172.17.18.116:6380@16380 slave 980182b3a13d4617fe9d757eb5c894ab2f2a070b 0 1598327172000 5 connected 980182b3a13d4617fe9d757eb5c894ab2f2a070b 172.17.19.51:6379@16379 master - 0 1598327174936 3 connected 10923-16383 e68cc0fcc2f27cf289e0606ca230864a3bddea0a 172.17.18.164:6379@16379 master - 0 1598327173000 2 connected 5461-10922 e031c734f1612f6b62389c362133e646946d3628 172.17.18.164:6380@16380 slave c246ddcbdf31625a21d24815952a6eb39cfb4dde 0 1598327172932 4 connected fa3d71f57cde8605bc41836168312a443cf8216f 172.17.19.51:6380@16380 slave e68cc0fcc2f27cf289e0606ca230864a3bddea0a 0 1598327173934 6 connected

4. 集群扩容

添加一主一从。

#cluster2上执行 mkdir -p /opt/app/redis-cluster #cluster137上执行 cd /opt/app/redis-cluster scp -r 6379/ root@cluster2:/opt/app/redis-cluster/6379 scp -r 6380/ root@cluster2:/opt/app/redis-cluster/6380

添加6379节点作为新节点启动并加入集群（需要删除掉dump.rdb和node.conf文件再启动，新增的redis节点不能有数据）

cd /opt/app/redis-cluster/6379/bin #启动redis ./redis-server redis.conf #加入集群 ./redis-cli --cluster add-node 172.17.19.2:6379 172.17.18.116:6379 [root@cluster2:/opt/app/redis-cluster/6379/bin]$ ./redis-cli --cluster add-node 172.17.19.2:6379 172.17.18.164:6379 >>> Adding node 172.17.19.2:6379 to cluster 172.17.18.164:6379 >>> Performing Cluster Check (using node 172.17.18.164:6379) M: e68cc0fcc2f27cf289e0606ca230864a3bddea0a 172.17.18.164:6379 slots:[5461-10922] (5462 slots) master 1 additional replica(s) S: e031c734f1612f6b62389c362133e646946d3628 172.17.18.164:6380 slots: (0 slots) slave replicates c246ddcbdf31625a21d24815952a6eb39cfb4dde M: c246ddcbdf31625a21d24815952a6eb39cfb4dde 172.17.18.116:6379 slots:[0-5460] (5461 slots) master 1 additional replica(s) M: 980182b3a13d4617fe9d757eb5c894ab2f2a070b 172.17.19.51:6379 slots:[10923-16383] (5461 slots) master 1 additional replica(s) S: fa3d71f57cde8605bc41836168312a443cf8216f 172.17.19.51:6380 slots: (0 slots) slave replicates e68cc0fcc2f27cf289e0606ca230864a3bddea0a S: cbfef021f44ead084cf61fad4d96814bbed19300 172.17.18.116:6380 slots: (0 slots) slave replicates 980182b3a13d4617fe9d757eb5c894ab2f2a070b [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered. >>> Send CLUSTER MEET to node 172.17.19.2:6379 to make it join the cluster. [OK] New node added correctly.

查看是否加入集群成功

[root@cluster2:/opt/app/redis-cluster/6379/bin]$ ./redis-cli -c 127.0.0.1:6379> get key:name -> Redirected to slot [50] located at 172.17.18.116:6379 "zhaoyun" 172.17.18.116:6379> cluster nodes c246ddcbdf31625a21d24815952a6eb39cfb4dde 172.17.18.116:6379@16379 myself,master - 0 1598357145000 1 connected 0-5460 791128e9780fa8b265bed63f445db7e0e4044304 172.17.19.2:6379@16379 master - 0 1598357147000 0 connected cbfef021f44ead084cf61fad4d96814bbed19300 172.17.18.116:6380@16380 slave 980182b3a13d4617fe9d757eb5c894ab2f2a070b 0 1598357145379 5 connected 980182b3a13d4617fe9d757eb5c894ab2f2a070b 172.17.19.51:6379@16379 master - 0 1598357146381 3 connected 10923-16383 e68cc0fcc2f27cf289e0606ca230864a3bddea0a 172.17.18.164:6379@16379 master - 0 1598357148384 2 connected 5461-10922 e031c734f1612f6b62389c362133e646946d3628 172.17.18.164:6380@16380 slave c246ddcbdf31625a21d24815952a6eb39cfb4dde 0 1598357145000 4 connected fa3d71f57cde8605bc41836168312a443cf8216f 172.17.19.51:6380@16380 slave e68cc0fcc2f27cf289e0606ca230864a3bddea0a 0 1598357147382 6 connected

hash槽重新分配（数据迁移） 添加完主节点是没有分配slot的，需要对主节点进行hash槽分配，这样该主节才可以存储数据。

给刚添加的6379节点分配槽 第一步：连接上集群（连接集群中任意一个可用结点都行）

执行命令， ./redis-cli –cluster reshard 172.17.19.2:6379

输出：

>>> Performing Cluster Check (using node 172.17.19.2:6379) M: 791128e9780fa8b265bed63f445db7e0e4044304 172.17.19.2:6379 slots: (0 slots) master M: 980182b3a13d4617fe9d757eb5c894ab2f2a070b 172.17.19.51:6379 slots:[10923-16383] (5461 slots) master 1 additional replica(s) S: e031c734f1612f6b62389c362133e646946d3628 172.17.18.164:6380 slots: (0 slots) slave replicates c246ddcbdf31625a21d24815952a6eb39cfb4dde S: fa3d71f57cde8605bc41836168312a443cf8216f 172.17.19.51:6380 slots: (0 slots) slave replicates e68cc0fcc2f27cf289e0606ca230864a3bddea0a M: c246ddcbdf31625a21d24815952a6eb39cfb4dde 172.17.18.116:6379 slots:[0-5460] (5461 slots) master 1 additional replica(s) M: e68cc0fcc2f27cf289e0606ca230864a3bddea0a 172.17.18.164:6379 slots:[5461-10922] (5462 slots) master 1 additional replica(s) S: cbfef021f44ead084cf61fad4d96814bbed19300 172.17.18.116:6380 slots: (0 slots) slave replicates 980182b3a13d4617fe9d757eb5c894ab2f2a070b [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered. How many slots do you want to move (from 1 to 16384)?

第二步：输入要分配的槽数量 ,分配3000个

How many slots do you want to move (from 1 to 16384)? 3000

输出：

What is the receiving node ID? 

第三步：输入接收槽的节点ID

What is the receiving node ID? 791128e9780fa8b265bed63f445db7e0e4044304

输出：

Please enter all the source node IDs. Type 'all' to use all the nodes as source nodes for the hash slots. Type 'done' once you entered all the source nodes IDs. Source node #1:

第四步：输入源节点ID,输入：all

Source node #1:all

输出：

Ready to move 3000 slots. Source nodes: M: 980182b3a13d4617fe9d757eb5c894ab2f2a070b 172.17.19.51:6379 slots:[10923-16383] (5461 slots) master 1 additional replica(s) M: c246ddcbdf31625a21d24815952a6eb39cfb4dde 172.17.18.116:6379 slots:[0-5460] (5461 slots) master 1 additional replica(s) M: e68cc0fcc2f27cf289e0606ca230864a3bddea0a 172.17.18.164:6379 slots:[5461-10922] (5462 slots) master 1 additional replica(s) Destination node: M: 791128e9780fa8b265bed63f445db7e0e4044304 172.17.19.2:6379 slots: (0 slots) master Resharding plan: Moving slot 5461 from e68cc0fcc2f27cf289e0606ca230864a3bddea0a Moving slot 5462 from e68cc0fcc2f27cf289e0606ca230864a3bddea0a Moving slot 5463 from e68cc0fcc2f27cf289e0606ca230864a3bddea0a Moving slot 5464 from e68cc0fcc2f27cf289e0606ca230864a3bddea0a Moving slot 5465 from e68cc0fcc2f27cf289e0606ca230864a3bddea0a Moving slot 5466 from e68cc0fcc2f27cf289e0606ca230864a3bddea0a Moving slot 5467 from e68cc0fcc2f27cf289e0606ca230864a3bddea0a Moving slot 5468 from e68cc0fcc2f27cf289e0606ca230864a3bddea0a Moving slot 5469 from e68cc0fcc2f27cf289e0606ca230864a3bddea0a Moving slot 5470 from e68cc0fcc2f27cf289e0606ca230864a3bddea0a Moving slot 5471 from e68cc0fcc2f27cf289e0606ca230864a3bddea0a Moving slot 5472 from e68cc0fcc2f27cf289e0606ca230864a3bddea0a Moving slot 5473 from e68cc0fcc2f27cf289e0606ca230864a3bddea0a Moving slot 5474 from e68cc0fcc2f27cf289e0606ca230864a3bddea0a Moving slot 5475 from e68cc0fcc2f27cf289e0606ca230864a3bddea0a

第五步：输入yes开始移动槽到目标节点id

Do you want to proceed with the proposed reshard plan (yes/no)? yes

输出：

Moving slot 985 from 172.17.18.116:6379 to 172.17.19.2:6379: Moving slot 986 from 172.17.18.116:6379 to 172.17.19.2:6379: Moving slot 987 from 172.17.18.116:6379 to 172.17.19.2:6379: Moving slot 988 from 172.17.18.116:6379 to 172.17.19.2:6379: Moving slot 989 from 172.17.18.116:6379 to 172.17.19.2:6379: Moving slot 990 from 172.17.18.116:6379 to 172.17.19.2:6379: Moving slot 991 from 172.17.18.116:6379 to 172.17.19.2:6379: Moving slot 992 from 172.17.18.116:6379 to 172.17.19.2:6379: Moving slot 993 from 172.17.18.116:6379 to 172.17.19.2:6379: Moving slot 994 from 172.17.18.116:6379 to 172.17.19.2:6379: Moving slot 995 from 172.17.18.116:6379 to 172.17.19.2:6379: Moving slot 996 from 172.17.18.116:6379 to 172.17.19.2:6379: Moving slot 997 from 172.17.18.116:6379 to 172.17.19.2:6379: Moving slot 998 from 172.17.18.116:6379 to 172.17.19.2:6379:

添加从节点 添加6380从节点，将6380作为6379的从节点。

命令：

./redis-cli --cluster add-node 新节点的ip和端口 旧节点ip和端口 --cluster-slave --cluster-master-id 主节点id ./redis-cli --cluster add-node 172.17.19.2:6380 172.17.19.2:6379 --cluster-slave --cluster-master-id 791128e9780fa8b265bed63f445db7e0e4044304

查看是否加入成功。

原文链接：https://blog.csdn.net/weixin_42496757/article/details/108372775