What are the redis cluster solutions?

The amount of Redis data is increasing day by day, and more and more companies are using it. It is not only used for caching, but also tends to store this piece. This will inevitably promote the development of clusters and various companies. I am also collecting cluster solutions suitable for myself. Currently, the following cluster architectures are commonly used in the industry. Most of them use sharding technology to solve a series of problems caused by the increase of single instance memory.

This article briefly introduces five solutions:

Official cluster solution

##twemproxy proxy solution

Sentinel mode

codis

Client sharding

Official cluser solution

Starting from redis version 3.0, redis-cluster cluster is supported. redis-cluster adopts a centerless structure. Each node saves data and the entire cluster status, and each node is connected to other nodes. redis-cluster is a server-side sharding technology.

redis-cluster architecture diagram

redis-cluster features:

Each node communicates with n-1 nodes. This It is called a cluster bus. They use a special port number, which is the external service port number plus 10000. Therefore, it is necessary to maintain the information of each node in this cluster, otherwise the entire cluster will be unavailable. A special binary protocol is used internally to optimize the transmission speed and bandwidth.

redis-cluster maps all physical nodes to [0,16383] slot (slot), and cluster is responsible for maintaining node--slot--value.

The cluster is pre-divided into 16384 buckets. When data needs to be inserted into the redis cluster, it is decided according to the value of CRC16(KEY) mod 16384 which bucket a key should be placed in.

The client is directly connected to the redis node. There is no need to connect to all nodes in the cluster, just connect to any available node in the cluster.

The redis-trib.rb script (rub language) is a cluster management tool, such as automatically adding nodes, planning slots, migrating data and a series of operations.

The fail of a node takes effect only when more than half of the nodes in the cluster detect failures.

The entire cluster is regarded as a whole. The client can connect to any node for operation. When the key operated by the client is not allocated on the node, redis will return a steering instruction to point to the correct node.

In order to increase the accessibility of the cluster, the officially recommended solution is to configure the node into a master-slave structure, that is, a master node and n slave nodes. If the master node fails, redis cluster will select one of the slave nodes to be promoted to the master node according to the election algorithm, and the entire cluster will continue to provide services to the outside world.

twemproxy proxy solution

twemproxy proxy architecture diagram:

Redis proxy middleware twemproxy is a kind of sharding that uses middleware technology. Twemproxy is between the client and the server. It processes the request from the client (sharding) and then forwards it to the real redis server at the back end. In other words, the client does not directly access the redis server, but indirectly accesses it through the twemproxy proxy middleware. Reduces the number of direct client connections to back-end servers, and supports horizontal expansion of server clusters.

The internal processing of twemproxy middleware is stateless, and it can be easily clustered itself, which avoids single points of stress or failure.

twemproxy, also known as nutcracker, originated from the lightweight proxy of redis and memcached clusters in the Twitter system.

Github source code address (https://github.com/twitter/twemproxy)

From the above architecture diagram, we can see that twemproxy is a single point, which can easily cause a lot of pressure on it. Therefore, keepalived is usually combined to achieve high availability of twemproy. At this time, usually only one twemproxy is working, and the other one is on standby. When one hangs up, VIP automatically drifts and the standby machine takes over the work. Regarding the usage of keepalived, you can check the information online.

Sentinel Mode

Sentinel Sentinel

Sentinel is a high-availability solution for Redis: a Sentinel system composed of one or more Sentinel instances can monitor any number of The master server and all slave servers under these master servers, and when the monitored master server goes offline, automatically upgrade a slave server under the offline master server to a new master server.

For example:

After Server1 goes offline:

Upgrade Server2 as the new master Server:

How Sentinel works

Each Sentinel sends messages to the Master, Slave and other Sentinel instances it knows about once per second A PING command.

If the time since the last valid reply to the PING command for an instance exceeds the value specified by the down-after-milliseconds option, the instance will be marked as subjectively offline by Sentinel.

If a Master is marked as subjective offline, all Sentinels that are monitoring the Master must confirm once per second that the Master has indeed entered the subjective offline state.

When a sufficient number of Sentinels (greater than or equal to the value specified in the configuration file) confirm that the Master has indeed entered a subjective offline state within the specified time range, the Master will be marked as objectively offline.

Under normal circumstances, each Sentinel will send INFO commands to all Masters and Slaves it knows once every 10 seconds.

When the Master is marked as objectively offline by Sentinel, the frequency of Sentinel sending INFO commands to all slaves of the offline Master will be changed from once every 10 seconds to once every second.

If there are not enough Sentinels to agree that the Master has been offline, the Master's objective offline status will be removed. If the Master returns a valid value to Sentinel's PING command again, the Master's subjective offline status will be removed.

codis

codis is a distributed Redis solution, open sourced by Wandoujia. For upper-layer applications, there is no obvious difference between connecting to codis proxy and connecting to the native redis server. The upper-layer The application can be used like a stand-alone redis. The bottom layer of codis will handle the forwarding of requests, non-stop data migration, etc. All the subsequent things are transparent to the previous client. You can simply think that the connection behind is A redis service with unlimited memory.

Client Sharding

The logic of partitioning is implemented on the client, and the client chooses which node to request. The solution can refer to consistent hashing. This solution is usually suitable for scenarios where users have complete control over the behavior of the client.

Summary: There is no best solution, only the most suitable solution.

For more Redis-related technical articles, please visit the Redis Tutorial column to learn!

The above is the detailed content of What are the redis cluster solutions?. For more information, please follow other related articles on the PHP Chinese website!