Learn more about master-slave replication in Redis

This article will give you an understanding of master-slave replication in Redis, introduce the basic configuration of master-slave, and the functions and principles of master-slave configuration. I hope it will be helpful to you!

Redis supports master-slave replication function, which can be done by executing slaveof (changed to replicaof after Redis5 version) or setting slaveof in the configuration file (changed to replicaof after Redis5 version) Turn on the copy function. [Related recommendations: Redis video tutorial]

• One master and two clusters

• One master Multi-slave

Master-slave basic configuration

Master Redis configuration

Master Redis configuration is basically not needed Modification, the key part is from Redis configuration

From Redis configuration

1. Copy a redis.conf file

2. Relevant configuration modifications

# salve的端口号
port 6380 

#把pid进程号写入pidfile配置的文件
pidfile /var/run/redis_6380.pid 

logfile "6380.log"  

#指定数据存放目录
dir /usr/local/redis‐5.0.3/data/6380 

#需要注释掉bind
#bind127.0.0.1（bind绑定的是自己机器网卡的ip，如果有多块网卡可以配多个ip，代表允许客户端通过机器的哪些网卡ip去访问，内网一般可以不配置bind，注释掉即可）

3. Configure master-slave replication

#从本机master6379的redis实例复制数据，Redis5.0之前使用slaveof
replicaof 192.168.0.60 6379

#配置从节点只读
replica‐read‐only yes

4. Start the slave node

redis‐server redis.conf

5. Connect the slave node

redis‐cli ‐p 6380

6. Test writing data on the 6379 instance to see whether the 6380 instance can synchronize the newly modified data in time

docker run  --name redis-6381 -v /Users/yujiale/docker/redis/conf/redis6381.conf:/etc/redis/redis.conf -v /Users/yujiale/docker/redis/conf/sentinel6381.conf:/etc/redis/sentine.conf -v /Users/yujiale/docker/redis/data6381:/data --network localNetwork --ip 172.172.0.14 -p 16381:6379 -d redis:6.2.6 redis-server /etc/redis/redis.conf --appendonly yes

The role of master-slave configuration

Separation of reading and writing

• One master and multiple slaves, master-slave synchronization
• The master is responsible Write, slave is responsible for reading
• Improve the performance and throughput of Redis
• Master-slave data consistency issue

Data disaster recovery

• The slave is the backup of the host
• When the host is down, the slave can read but not write
• By default, after the host goes down, the slave cannot be used by the host
• Sentinel can realize master-slave switching and achieve high availability

Redis master-slave working principle

Full replication of master-slave replication

Only the first time a slave Redis connects to the main Redis, a full copy occurs. If it is a short-term resumption, it may be a full copy or a partial copy.

• Flowchart

1. Establish a long Socker connection with the main Redis

slaver establishes a socket connection with the master

slaver associated file event processor

• This processor receives RDB files (full copy) and receives Master propagation Coming write command (incremental copy)

• After the master server accepts the slave server Socket connection, it creates the corresponding client status. It is equivalent to the slave server being the client of the master server.

• Send ping command

  • Slaver sends ping command to Master

    • 1. Check the read and write status of the socket

    • 2. Check whether the Master can handle it normally

  • Master's response:

    • 1. Send "pong", indicating that it is normal

    • 2. Return an error, indicating that the Master is not normal

    • 3. timeout, indicating network timeout

##Permission verification

After the master and slave are connected normally, perform permission verification

The master has not set a password (requirepass=""), and the slave does not need to set a password (masterauth="" ”)

The master sets the password (requirepass!=""), the slave needs to set the password (masterauth=the value of the master's requirepass)

Or the slave sends the password to the master through the auth command

2. The main Redis receives the PSYNC command

After the main Redis receives the PSYNC command, executing the bgsave command will generate the latest rdb snapshot,

3. The master Redis sends the rdb snapshot to the slave Redis

When the master Redis sends the rdb snapshot to the slave Redis, the master will continue to receive the client's request, and it will The request cache that may modify the data set is stored in the relp buffer cache in memory

• Synchronization snapshot phase: Master creates and sends snapshot RDB to Slave, and Slave loads and parses the snapshot. The Master also stores new write commands generated during this phase into the buffer.

4. Receive the rdb snapshot from the node

After receiving the rdb snapshot from the node, clear the old data and load the rdb file

5. Master Redis sends the buffer cache file to slave Redis

Synchronous write buffer stage: Master synchronizes the write operation command stored in the buffer to Slave.

6. Receive the buffer cache file from the node

Receive the buffer cache file from the node and load the buffer cache file into the memory

7. The master Redis continuously sends commands to the slave node through the Socker long connection

The slave Redis receives the command sent by the master Redis and executes the current command

Overview

If you configure a slave for the master, regardless of whether the slave connects to the master for the first time, it will send a PSYNC command to the master to request copy data. After receiving the PSYNC command, the master will perform data persistence in the background and generate the latest RDB snapshot file through bgsave. During the persistence period, the master will continue to receive client requests, and it will cache these requests that may modify the data set in memory. When the persistence is completed, the master will send the RDB file data set to the slave, and the slave will persist the received data to generate RDB, and then load it into the memory. Then, the master sends the previously cached commands in memory to the slave. When the connection between the master and the slave is disconnected for some reason, the slave can automatically reconnect to the master. If the master receives multiple slave concurrent connection requests, it will only persist once, not once for each connection, and then Then send this persistent data to multiple concurrently connected slaves.

Partial copy of master-slave copy

The general process is similar to full copy, so I won’t explain too much

Brief description

When the master and slave are disconnected and reconnected, the entire data will generally be copied. However, starting from redis version 2.8, redis uses the command PSYNC that can support partial data replication to synchronize data with the master. The slave and master can only perform partial data replication (resumed transmission) after the network connection is disconnected and reconnected. The master will create a cache queue for copying data in its memory to cache the data for the most recent period. The master and all its slaves maintain the copied data subscript offset and the master's process id. Therefore, when the network connection is disconnected, Afterwards, the slave will request the master to continue the unfinished replication, starting from the recorded data index. If the master process ID changes, or the slave node data offset is too old and is no longer in the master's cache queue, then a full data copy will be performed. Master-slave replication (partial replication, breakpoint resume) flow chart:

Incremental synchronization of master-slave replication

• Redis incremental synchronization It mainly refers to the process in which the write operations that occur on the Master are synchronized to the Slave when the Slave completes initialization and starts to work normally.

• Normally, every time the Master executes a write command, it will send the same write command to the Slave, and then the Slave will receive and execute it.

Heartbeat detection of master-slave replication

1. Detect the connection status of master-slave

Detect the network connection status of the master and slave servers. By sending the INFO replication command to the master server, you can list the slave servers. You can see how many seconds have passed since the last command was sent to the master. The value of lag should jump between 0 or 1. If it exceeds 1, it means that the connection between the master and the slave is faulty.

2. Auxiliary implementation of min-slaves

Redis can be configured to prevent the main server from executing the write command min-slaves-to-write 3 ( min-replicas-to-write 3) min-slaves-max-lag 10 (min-replicas-max-lag 10) The above configuration means: the number of slave servers is less than 3, or the delay of three slave servers (lag ) values ​​are greater than or equal to 10 seconds, the master server will refuse to execute the write command. The delay value here is the lag value of the INForeplication command above.

3. Detect command loss

If the write command transmitted from the master server to the slave server is lost halfway due to a network failure, then when the slave server sends REPLCONF to the master server When executing the ACK command, the master server will find that the current replication offset of the slave server is less than its own replication offset. Then the master server will find the missing data of the slave server in the replication backlog buffer based on the replication offset submitted by the slave server. data and resends the data to the slave server. (Reissue) The network is continuously synchronized incrementally: the network is disconnected, and when connected again

How to judge full replication or partial replication

After the client sends saveof, the master node will determine whether to replicate for the first time, and if so, proceed Full copy, if the runid offset is not used to determine whether it is consistent, if consistent, partial copy will be performed, otherwise full copy will be performed.

For more programming related knowledge, please visit: Programming Video! !

The above is the detailed content of Learn more about master-slave replication in Redis. For more information, please follow other related articles on the PHP Chinese website!