An article to talk about the Cluster cluster in Redis

This article will introduce you to the Cluster cluster in Redis, take a look at TCP ports, data sharding, and learn how to use docker to build a Redis cluster with three masters and three slaves.

Redis Cluster (Redis Cluster)

Redis Cluster is Redis Distributed implementation. When we transfer data to Redis Cluster, the data will be automatically fragmented and stored on each Redis node. [Related recommendations: Redis Video Tutorial]

Compared with single-point Redis, Redis Cluster can continue to run when some nodes fail or cannot communicate. However, if there is a major server failure (for example, more than half of the servers are unavailable), the cluster will stop functioning.

Redis Cluster TCP Port

Each Redis cluster node needs to open two TCP connections. One is the regular TCP port used to serve clients, which defaults to 6379. The second port is used for the cluster bus, with the default setting of 16379, a binary protocol node-to-node communication channel. Nodes utilize the cluster bus for failure detection, configuration updates, failover authorization, etc.

• Clients should never try to communicate with the cluster bus port, but always with the normal Redis command port, but make sure to open both ports in the firewall at the same time, otherwise the Redis cluster node will Communication is not possible. The offset of the command port and cluster bus port is fixed and is always 10000.
• If both TCP ports are not opened at the same time, the cluster will not work properly.
• The cluster bus uses a different binary protocol for node-to-node data exchange. It is more suitable for exchanging information between nodes with very little bandwidth and processing time.

Redis Cluster data sharding

Redis Cluster does not use consistent hashing, but uses a hash slot. hash_slot stuff.

There are 16384 hash slots in the Redis cluster. When we store a pair of Key-Value in Redis, we need to calculate the hash slot for a given Key. The method is to first calculate the CRC16 of the Key, and then take the calculated result modulo 16384:

hash_slot = CRC16(key) mod 16384

Everyone in the Redis cluster Redis nodes are each responsible for a subset of hash slots, so if you have a cluster of 3 nodes where:

• Node A contains hash slots from 0 to 5500.
• Node B contains hash slots from 5501 to 11000.
• Node C contains hash slots from 11001 to 16383.

This makes it easy to add and remove nodes in the cluster. For example, if I want to add a new node D, I need to move some hash slots from nodes A, B, C to D. Similarly, if I want to remove node A from the cluster, I just move the hash slots served by A to B and C. When node A is empty, I can completely remove it from the cluster.

Because moving a hash slot from one node to another does not require downtime, adding or removing or changing the proportion of hash slots held by a node does not require any downtime.

Next we use docker to build a Redis cluster with three masters and three slaves.

Use Docker to build RedisCluster

Redis configuration

port ${PORT}                                       ##节点端口
protected-mode no                                  ##开启集群模式
cluster-enabled yes                                ##cluster集群模式
cluster-config-file nodes.conf                     ##集群配置名
cluster-node-timeout 5000                          ##超时时间
cluster-announce-ip 192.168.1.XX                   ##实际为各节点网卡分配ip  先用一个ip代替
cluster-announce-port ${PORT}                      ##节点映射端口
cluster-announce-bus-port 1${PORT}                 ##节点总线端口
appendonly yes                                     ##持久化模式

Create a custom network

docker network create redis-net

Custom path

mkdir -p /usr/redis_cluster
cd /usr/redis_cluster

Generate conf and data targets under the custom path, and generate configuration information

for port in `seq 6001 6006`; do
    mkdir -p ./${port}/conf
    touch ./${port}/conf/redis.conf 
    mkdir -p ./${port}/data 
    echo "port ${port}" >>./${port}/conf/redis.conf
    echo "protected-mode no" >>./${port}/conf/redis.conf
    echo "cluster-enabled yes" >>./${port}/conf/redis.conf
    echo "cluster-config-file nodes.conf" >>./${port}/conf/redis.conf
    echo "cluster-node-timeout 5000" >>./${port}/conf/redis.conf
    echo "cluster-announce-ip 192.168.1.XX" >>./${port}/conf/redis.conf
    echo "cluster-announce-port ${port}" >>./${port}/conf/redis.conf
    echo "cluster-announce-bus-port 1${port}" >>./${port}/conf/redis.conf
    echo "appendonly yes" >>./${port}/conf/redis.conf
done

• cluster-announce-ip The IP in 192.168.1.XX must be the IP for communication between containers, and can be viewed in the previously added network.
• A total of 6 folders are generated, from 6001 to 6006. Each folder contains data and conf folders, and conf contains the redis.conf configuration file.

Start the Redis container

for port in `seq 6001 6006`; do \
  docker run -d --privileged=true -p ${port}:${port} -p 1${port}:1${port}\
  -v $PWD/${port}/conf/redis.conf:/usr/local/etc/redis/redis.conf \
  -v $PWD/${port}/data:/data \
  --restart always --name redis-${port} --net redis-net \
  redis:5.0.5 redis-server /usr/local/etc/redis/redis.conf; \
done

Start the cluster

# 进入任意Redis容器
docker exec -it redis-6001 /bin/bash
# 初始化Redis集群命令
redis-cli --cluster create 172.19.0.2:6601 172.19.0.3:6602 172.19.0.4:6603 172.19.0.5:6604 172.19.0.6:6605 172.19.0.7:6606 --cluster-replicas 1

• After the creation is successful, we can use redis The -cli command connects to one of the Redis services.

# 单机模式启动
redis-cli -h 127.0.0.1 -p 6001
# 集群模式启动
redis-cli -c -h 127.0.0.1 -p 6001

• Afterwards, you can view the node information through the cluster nodes command and find that it meets the expectations of 3 masters and 3 slaves

Reference documentation

• Redis official documentation

More programming For related knowledge, please visit: programming video! !

The above is the detailed content of An article to talk about the Cluster cluster in Redis. For more information, please follow other related articles on the PHP Chinese website!