How to use Redis's expiration strategy and memory elimination strategy

1 Set the key with expiration time

expire key seconds
时间复杂度：O（1）

Set the expiration time of key. After timeout, the key will be automatically deleted. In Redis terminology the associated timeout for a key is volatile.

After timeout, it will only be cleared when DEL, SET, or GETSET is executed on key. This means that conceptually all operations that change the key without replacing it with a new value will keep the timeout unchanged. For example, use INCR to increment the value of key, execute LPUSH to push the new value into the list or use HSET to change the field of the hash, These operations all leave the timeout unchanged.

• Use the PERSIST command to clear the timeout and make it a permanent key

• If key is modified by the RENAME command, the related timeout period will be transferred to the new key

• ##if

  key is modified by the RENAME command. For example, Key_A originally existed, and then the RENAME Key_B Key_A command is called. At this time, the original Key_A## is ignored. # Whether it is permanent or set to timeout will be overwritten by the validity status of Key_B

Note that EXPIRE/PEXPIRE is called with a non-positive timeout or with a past time EXPIREAT/PEXPIREAT will cause the key to be deleted rather than expired (so the key event emitted will be del, not expired).

1.1 Refresh the expiration time

Performing the

EXPIRE

operation on the key that already has an expiration time will update its expiration time. There are many applications with this business scenario, such as session recording. 1.2 Differences in Redis before 2.1.3

In Redis versions before 2.1.3, changing a key with an expired set using the command that changes its value has the effect of completely deleting the key. This semantics is required due to limitations in the now-fixed replication layer.

EXPIRE will return 0 and will not change the timeout for keys with a timeout set.

1.3 Return value

• 1

  If the expiration time is successfully set.

• #0

  If key does not exist or the expiration time cannot be set.

1.4 Example

Suppose there is a Web service that is interested in the latest N pages recently visited by the user, so that each adjacent Page view is no more than 60 seconds after the previous page. Conceptually, think of this set of page views as a navigation session for the user, which might contain interesting information about the products they are currently looking for so that you can recommend related products.

This pattern can be easily modeled in Redis using the following strategy: Every time the user executes a page view, you call the following command:

MULTI
RPUSH pagewviews.user:<userid> http://.....
EXPIRE pagewviews.user:<userid> 60
EXEC</userid></userid>

If the user is idle for more than 60 seconds, then Delete the key and only record subsequent page views that differ by less than 60 seconds. This mode is easily modified to use INCR instead of a list using RPUSH.

1.5 Key with expiration time

Normally, a Redis key is created without an associated survival time. The key will persist unless the user deletes it explicitly (such as the DEL command). Use the EXPIRE command to associate an expired item with a given key, but this will cause the key to occupy additional memory. Redis will automatically delete keys with expired sets after a specified time to ensure that the data does not expire. The critical time to live can be updated or completely removed using the EXPIRE and PERSIST commands (or other strict commands).

1.6 Expiration precision

The expiration time in Redis 2.4 may not be precise, fluctuating between 0 and 1 second. Since Redis 2.6, the expiration error ranges from 0 to 1 milliseconds.

1.7 Expiration and persistence

Store the key of expired information as an absolute Unix timestamp. The millisecond-level storage method is suitable for Redis version 2.6 and higher. This means that time is flowing even when the Redis instance is not active. For expiration to work well, the computer time must be stabilized. If you move an RDB file from two machines that have large desyncs in their clocks, interesting things may happen (like loading all keys that are out of date). Even when running an instance, the computer clock is always checked, for example, if you set a key to 1000 seconds and then set the computer time 2000 seconds in the future, the key will expire immediately instead of lasting 1000 seconds.

2 How to expire keys in Redis

There are two ways to expire keys: passive way - lazy deletion, active way - regular deletion.

2.1 Lazy deletion

When the client tries to access the key, the key will expire passively, that is, Redis will check whether the key has an expiration time set, and if it expires, it will be deleted. Return nothing. Redis will not automatically delete the key, but when querying the key, Redis will lazily check whether it has been deleted. This is similar to Spring's delayed initialization.

当然，这是不够的，因为有过期的key，永远不会再访问。无论如何，这些key都应过期，因此请定期 Redis 在具有过期集的key之间随机测试几个key。已过期的所有key将从key空间中删除。

2.2 定期删除

具体来说，如下 Redis 每秒 10 次：

1. 测试 20 个带有过期的随机键

2. 删除找到的所有已过期key

3. 如果超过 25% 的key已过期，从步骤 1 重新开始

这是一个微不足道的概率算法，基本上假设我们的样本代表整个key空间，继续过期，直到可能过期的key百分比低于 25%。在任何特定时刻，已失效的最大键数等于每秒最大写入操作数除以4，这是由内存使用所决定的。

2.3 在复制链路和 AOF 文件中处理过期的方式

为了在不牺牲一致性的情况下获得正确行为，当key过期时，DEL 操作将同时在 AOF 文件中合成并获取所有附加的从节点。这样做的好处是能够将过时的处理过程集中在主节点中，避免出现一致性错误的可能性。

但是，虽然连接到主节点的从节点不会独立过期key（但会等待来自master的 DEL），但它们仍将使用数据集中现有过期的完整状态，因此，当选择slave作为master时，它将能够独立过期key，完全充当master。

由于您没有及时查找和删除大量过期key，这些过期key在Redis中堆积，导致内存严重耗尽

因此还需有内存淘汰机制！

3 内存淘汰

3.1 内存淘汰策略

noeviction（Redis默认策略）

写请求无法继续服务 (DEL 请求除外)，但读请求可以继续进行。这样 可以保证不会丢失数据，但是会让线上的业务不能持续进行。

• config.c

createEnumConfig("maxmemory-policy", NULL, 
	MODIFIABLE_CONFIG, maxmemory_policy_enum, 
		server.maxmemory_policy, 
			MAXMEMORY_NO_EVICTION, NULL, NULL),

allkeys-random

当内存不足以容纳新写入数据时，在键空间中，随机移除某key。凭啥随机呢，至少也是把最近最少使用的key删除。

allkeys-lru(Least Recently Used)

当内存不足以容纳新写入数据时，在键空间中，移除最近最少使用的key，没有设置过期时间的 key 也会被淘汰。

allkeys-lfu(Least Frequently Used)

LRU的关键是看页面最后一次被使用到发生调度的时间长短，而LFU关键是看一定时间段内页面被使用的频率。

volatile-lru

优先淘汰最少使用的 key，其中包括设置了过期时间的 key。 没有设置过期时间的 key 不会被淘汰，这样可以保证需要持久化的数据不会突然丢失。与allkey-lru不同，这种策略仅淘汰过期的键集合。

volatile-lfu

volatile-random

淘汰的 key 是过期 key 集合中随机的 key。

volatile-ttl

淘汰的策略不是 LRU，而是 key 的剩余寿命 ttl 的值，ttl 越小越优先被淘汰。

volatile-xxx 策略只会针对带过期时间的 key 进行淘汰，allkeys-xxx 策略会对所有的 key 进行淘汰。

• 如果你只是拿 Redis 做缓存，那应该使用 allkeys-xxx，客户端写缓存时不必携带过期时间。

• 如果你还想同时使用 Redis 的持久化功能，那就使用 volatile-xxx 策略，这样可以保留没有设置过期时间的 key，它们是永久的 key 不会被 LRU 算法淘汰。

3.2 手写LRU

确实有时会问这个，因为有些候选人如果确实过五关斩六将，前面的问题都答的很好，那么其实让他写一下LRU算法，可以考察一下编码功底

你可以现场手写最原始的LRU算法，那个代码量太大了，不太现实

public class LRUCache<k> extends LinkedHashMap<k> {
    
private final int CACHE_SIZE;

    // 这里就是传递进来最多能缓存多少数据
    public LRUCache(int cacheSize) {
    	//  true指linkedhashmap将元素按访问顺序排序
        super((int) Math.ceil(cacheSize / 0.75) + 1, 0.75f, true);
        CACHE_SIZE = cacheSize;
    }

    @Override
    protected boolean removeEldestEntry(Map.Entry eldest) {
    	 // 当KV数据量大于指定缓存个数时，就自动删除最老数据
        return size() > CACHE_SIZE;
    }

}</k></k>

The above is the detailed content of How to use Redis's expiration strategy and memory elimination strategy. For more information, please follow other related articles on the PHP Chinese website!