How to expand redis

A classmate in the team used Redis as a cache in his project and stored hotspot data in Redis. In order to improve performance, the pipeline method is used when writing Redis. When used normally, the performance and resource usage of Redis can meet the project requirements. However, when the number of visits increases, the QPS of Redis can still meet the requirements, but the CPU usage is high. has reached 90%, usually only 30%. As we all know, Redis is a single process and can only occupy 1 CPU core. When it is full, it will be 100%. It cannot use the multi-core of the machine. When the CPU reaches 100% When, it will inevitably cause a performance bottleneck. How to deal with it?

Recommended: "Redis Video Tutorial"

Option 1:

The first thing that comes to mind is to increase the number of Redis servers, perform a hash operation on the stored keys on the client, and store them in different Redis servers. When reading, the same hash operation is also performed to find the corresponding Redis server. Solve the problem, but the disadvantages:

First, the client needs to change the code;

Second, the client needs to remember the addresses of all Redis servers;

This solution can be used, but can it be expanded without changing the code?

Option 2:

Build a cluster. Since the version used by the Redis server is lower than 3.0, it does not support clusters. You can only use a proxy, so I thought of the famous Redis proxy twemproxy.

The performance of twemproxy is also very good. Although it is a proxy, its impact on access performance is very small. Even the Redis author recommends it.

twemproxy is easy to use. A novice can learn to use it in less than an hour, and the key is that there is no need to change the client code. It supports almost all Redis commands and pipeline operations. You only need to change the client code. The Redis IP and PORT configured in the configuration file are changed from the original Redis IP and Port to the IP and PORT of the twemproxy service.

The client does not need to consider hash issues, twemproxy will do these, and the client is just like operating a Redis.

The word "almost" is used above because some commands, such as "keys *" are not supported

We quickly deployed twemproxy and the four following Redis machines. The test found that the CPU usage of the following four Redis units dropped, but a new problem came, twemproxy is also a single process! The performance bottleneck comes to twemproxy again!

Option 3:

Access to Redis is divided into writing and reading, similar to producers and consumers. After careful analysis, it is found that there is less writing and relatively less reading. More, this can separate reading and writing, writing to the primary, and reading from the backup. The situation encountered happens to be that reading and writing are two services. To achieve separation of reading and writing, just change the configuration information. It can be done very simply, thus dispersing the pressure on the main Redis.

The access pressure on Redis has improved here, but it is not a long-term solution. For example, when events are held and the amount of data increases, there will still be performance risks.

The final method adopted is comprehensive plan two and three, as shown in the figure below:

This method has minimal changes to existing services and can be effective The problem of alleviating redis pressure

The hash algorithm used by twemproxy on the producer side and the consumer side must be consistent, otherwise the key will not be found.

If plan 1 is also added, it will be more complicated and will not be used for the time being.

The above is the detailed content of How to expand redis. For more information, please follow other related articles on the PHP Chinese website!