What is the concept of Redis cache penetration and cache avalanche

1. Cache Penetration

1. Concept

The concept of cache penetration is very simple. The user wants to query a data and finds the redis memory The database does not have it, that is, the cache does not hit, so the persistence layer database is queried. It was found that there was none, so this query failed. When there are many users, the cache does not hit, so they all request the persistence layer database. When cache penetration occurs, the persistence layer database will bear a huge burden.

You need to pay attention to the difference between cache breakdown and cache breakdown. Cache breakdown refers to a key that is very hot and is constantly carrying large concurrency. Large concurrency concentrates on accessing this point. When The moment this key expires, the continuous large concurrency breaks through the cache and directly requests the database, which is like cutting a hole in a barrier.

There are actually many solutions to avoid cache penetration. Several are introduced below.

2. Solution

(1) Bloom filter

According to statistics, spam websites and normal websites around the world There are billions of websites in total, and Bloom filters are a data structure that can be applied to this scale of data. Using Bloom filters avoids the need for Internet police to compare spam websites in the database one by one. Suppose we store 100 million spam website addresses.

You can start with 100 million binary bits, and then the network police use eight different random number generators (F1, F2, …, F8) to generate eight information fingerprints (f1, f2, …, f8) . Next, a random number generator G is used to map these eight information fingerprints to eight natural numbers g1, g2, …, g8 from 1 to 100 million. Finally, set the binary values ​​of these eight positions to one. The process is as follows:

One day the Internet police found a suspicious website and wanted to determine whether it was the XX website. First, the suspicious website was hashed Maps to 8 points on a 100 million bit array. If there is a point that is not 1, then you can be sure that the element is not in the set.

So how does this Bloom filter solve cache penetration in redis? It's very simple. First of all, all possible query parameters are stored in hash form. When the user wants to query, they use the Bloom filter to find that they are not in the collection, and then discard them directly without querying the persistence layer.

This form is very simple.

2. Cache empty objects

When the storage layer misses, even the returned empty object will be cached, and an expiration time will be set. Accessing this data will be obtained from the cache, protecting the back-end data source

But this method will have two problems:

1. Since null values ​​may occupy many key positions in the cache, the cache requires more space to store more key-value pairs

2. Even for null values Even if the expiration time is set, the data in the cache layer and storage layer will still be inconsistent for a period of time, which will have an impact on businesses that need to maintain consistency.

2. Cache avalanche

1. Concept

Cache avalanche means that there is an error in the cache layer and it cannot work properly. Therefore, all requests are sent to the storage layer, causing the call volume of the storage layer to increase dramatically and possibly causing it to crash.

2. Solution

(1) redis high availability

The meaning of this idea is that since redis may hang up, then I add a few more redis so that the others can continue to work after one hangs up. In fact, it is a cluster.

(2) Current limiting downgrade

The idea of ​​this solution is to use locking or queue methods to control the threads that read the database and write to the cache after the cache expires. quantity. For example, only one thread is allowed to query data and write cache to operate a certain key, and the other threads need to wait.

(3) Data warm-up

The meaning of data warm-up is that before formal deployment, I first access the possible data in advance, so that some parts may be accessed in large quantities. The data will be loaded into the cache. Manually trigger the cache loading of different keys and set different expiration times to try to balance the cache invalidation time and prevent a large number of accesses from occurring at the same time.

The above is the detailed content of What is the concept of Redis cache penetration and cache avalanche. For more information, please follow other related articles on the PHP Chinese website!