An article to talk about the expiration strategy in Redis

This article will introduce you to the expiration strategy in Redis, and see how to implement the lazy deletion strategy and regular deletion strategy. I hope it will be helpful to you!

Save the expiration time

Redis can set the expiration time for each key, and will store each key with an expiration time set. into a separate dictionary. [Related recommendations: Redis Video Tutorial]

typedef struct redisDb { 
int id; //id是数据库序号，为0-15(默认Redis有16个数据库) 
long avg_ttl; //存储的数据库对象的平均ttl(time to live)，用于统计 
dict *dict; //存储数据库所有的key-value 
dict *expires; //存储key的过期时间 
dict *blocking_keys;//blpop 存储阻塞key和客户端对象 
dict *ready_keys;//阻塞后push 响应阻塞客户端 存储阻塞后push的key和客户端对象 dict *watched_keys;//存储watch监控的的key和客户端对象 
} redisDb;

dict is used to maintain all Key-Value key-value pairs contained in a Redis database, expires is used to maintain settings in a Redis database The key of the expiration time (that is, the mapping of key and expiration time). Note that the expiration time here is expressed in millisecond timestamps. For example, if 2022-01-02 22:45:02 expires, the value is 1641134702000

When we use the expire command to set a key When the expiration time is reached, Redis first searches the dict dictionary table to see if the key to be set exists. If it exists, the key and expiration time are added to the expires dictionary table.

When we use the setex command to insert data into the system, Redis first adds Key and Value to the dictionary table dict, and then adds Key and expiration time to the dictionary table expires. Note that setex can only be used for strings.

To put it simply, the key with the expiration time set and the specific expiration time are all maintained in the expires dictionary table.

Set the expiration time

Usage of expire

The expire command is used as follows: expire key ttl (unit seconds)

127.0.0.1:6379> expire name 2 #2秒失效 
(integer) 1 
127.0.0.1:6379> get name 
(nil) 
127.0.0.1:6379> set name zhangfei 
OK 
127.0.0.1:6379> ttl name #永久有效 
(integer) -1 
127.0.0.1:6379> expire name 30 #30秒失效 
(integer) 1 
127.0.0.1:6379> ttl name #还有24秒失效 
(integer) 24 
127.0.0.1:6379> ttl name #失效 
(integer) -2

Redis has four different commands that can be used to set the key's lifetime (how long the key can live) or expiration time (when the key will be deleted):

expire command is used Set the key's lifetime to ttl seconds

pexpire command is used to set the key's lifetime to ttlmilliseconds

expireat command Used to set the expiration time of the key key to the seconds timestamp specified by timestamp

pexpireat command is used to set the expiration time of the key key to the milliseconds specified by timestamp Number of timestamps

Note that the final implementation of expire, pexpire, and expireat are all implemented through pexpireat, which means that no matter which command is executed by the client, Redis will convert it into a pexpireat command for execution. Therefore, the time stored in the expires dictionary is the expiration time of the key represented by the millisecond timestamp.

Expiration Policy

If a key expires, when will it be deleted?

There are three expiration strategies

• Timed deletion: While setting the expiration time of the key, create a timer so that the timer can execute the deletion immediately when the expiration time of the key comes. key deletion operation. (Create timer deletion)
• Lazy deletion: Leave the expiration of the key alone, but every time you obtain the key from the key space, check whether the obtained key has expired. If it has expired, Delete the key; if it has not expired, return the key. (Delete when using)
• Regular deletion: Every once in a while, the program checks the database and deletes expired keys. There are algorithmic decisions as to how many expired keys to delete and how many databases to check. (Regular scanning and deletion)

Regular deletion

• Advantages

1. Yes The most memory-friendly: By using a timer, you can ensure that expired keys will be deleted as quickly as possible, freeing up the occupied memory

• Disadvantages

1. Most efficient for the CPU Unfriendly: When there are many expired keys, deleting expired keys may take up a considerable part of the CPU time, affecting the server's response time and throughput.

Lazy deletion

• Advantages

1. The most CPU-friendly: the key will only expire when the key is removed The key is checked, that is, the CPU does not need to be scanned regularly, and there is no need to create a large number of timers.

• Disadvantages

1. The least friendly to memory: If a key has expired but will not be accessed later, it will remain in the database. If there are too many such keys, it will undoubtedly occupy a lot of memory.

Periodic deletion

Periodic deletion is a compromise between the above scheduled deletion and lazy deletion.

• Advantages

1. Periodic deletion performs an expired key operation every once in a while, and reduces the impact of deletion operations on CPU time by limiting the duration and frequency of deletion operations. Impact.

2. By deleting expired keys, the memory waste caused by expired keys can be effectively reduced.

• Disadvantages It is difficult to determine the duration and frequency of deletion operations

1. If the deletion operation is performed too frequently or takes too long, the periodic deletion strategy will degenerate into scheduled deletion, which will take up too much CPU execution time.

2. If the deletion operation takes too little time, or the execution time is too short, the regular deletion strategy will be the same as lazy deletion, resulting in a waste of memory.

Redis expiration strategy

Redis uses two strategies: lazy deletion and regular deletion: use these two strategies by configuring them properly With this strategy, the server can strike a good balance between rational use of CPU time and avoiding wasted memory space.

Implementation of lazy deletion strategy

The lazy deletion strategy of expired keys is implemented by the db.c/expireIfNeeded function. All reading and writing databases Before the Redis command is executed, the expireIfNeed function will be called to check the input key:

• If the key has expired, then the expireIfNeeded function will delete the key
• If the key has not expired, then the expireIfNeeded function will not Do the operation

The command calls expireIfNeeded function process as shown below

In addition, because each accessed key may be deleted, each command Both must be able to handle both the presence and absence of keys. The following figure shows the execution process of the get command

Implementation of regular deletion strategy

Periodic expiration of expired keys The deletion strategy is implemented by the redis.c/activeExpireCycle function. Whenever the Redis server periodically operates the redis.c/serverCron function, the activeExpireCycle function will be called. It traverses each database in the server multiple times within a specified time.

Redis performs 10 expiration scans per second by default. Expiration scans do not traverse all keys in the expiration dictionary, but adopt a simple greedy strategy. The steps are as follows.

(1) Randomly select 20 keys from the expired dictionary.

(2) Delete the expired keys among these 20 keys.

(3) If the proportion of expired keys exceeds 1/4, repeat step (1). At the same time, in order to ensure that expired scanning does not over-cycle and cause the process to freeze, the algorithm also increases the upper limit of the scanning time, which will not exceed 25ms by default.

Suppose all the keys in a large Redis instance expire at the same time, what will happen?

Consumption of cpu

Redis The expired dictionary will continue to be scanned (looped multiple times) until the expired keys in the expired dictionary become sparse, and then it will stop (the number of loops is significantly reduced).

Resulting in request lag or timeout

When the client request arrives, if the server happens to enter the expired scanning state, the client's request will wait for at least 25ms. Processing, if the client sets the timeout time relatively short, such as 10ms, then a large number of connections will be closed due to timeout, and many exceptions will occur on the business end

So be sure to pay attention to expiration Time, if a large number of keys expire, a random range must be set for the expiration time, and they cannot all expire at the same time.

For more programming-related knowledge, please visit: Introduction to Programming! !

The above is the detailed content of An article to talk about the expiration strategy in Redis. For more information, please follow other related articles on the PHP Chinese website!