Before talking about distributed lock, it is necessary to explain why it is neededDistributed lock.
Compared with distributed locks, stand-alone locks. When writing multi-threaded programs, we avoid data problems caused by operating a shared variable at the same time. We usually use a lock to mutually exclude each other to ensure the correctness of shared variables. property, its scope of use is within the same process. If there are multiple processes that need to operate a shared resource at the same time, how can they be mutually exclusive? Today's business applications are usually microservice architecture, which also means that one application will deploy multiple processes. If multiple processes need to modify the same row of records in MySQL, in order to avoid dirty data caused by out-of-order operations, distribution needs to be introduced at this time. The style is locked.
#If you want to implement distributed locks, you must use an external system. All processes go to this system to apply for locks. This external system must be mutually exclusive, that is, if two requests arrive at the same time, the system will only successfully lock one process, and the other process will fail. This external system can be a database, Redis or Zookeeper, but in order to pursue performance, we usually choose to use Redis or Zookeeper.
Redis can be used as a shared storage system, and multiple clients can share access, so it can be used to save distributed locks. Moreover, Redis has high read and write performance and can handle high-concurrency lock operation scenarios. The focus of this article is to introduce how to use Redis to implement distributed locks, and discuss the problems that may be encountered during the implementation process.
As a shared storage system in the implementation of distributed locks, Redis can use key-value pairs to save lock variables and receive and process them. Operation requests for locking and releasing locks sent by different clients. So, how are the key and value of the key-value pair determined? We need to give the lock variable a variable name and use this variable name as the key of the key-value pair, and the value of the lock variable is the value of the key-value pair. In this way, Redis can save the lock variable, and the client can Lock operations can be implemented through Redis command operations.
To implement distributed locks, Redis must have mutual exclusion capabilities. You can use the SETNX command, which means SET IF NOT EXIST, that is, if the key does not exist, its value will be set, otherwise nothing will be done. A distributed lock is implemented by having two client processes execute the command mutually exclusive.
The following shows the operation process of Redis using key/value pairs to save lock variables and two clients requesting locks at the same time.
#After the locking operation is completed, the client that has successfully locked can operate the shared resources, for example, modify a certain row of data in MySQL. After the operation is completed, the lock must be released in time to give latecomers the opportunity to operate the shared resources. How to release the lock? Just use the DEL command to delete this key. The logic is very simple. The overall process written in pseudo code is as follows.
// 加锁 SETNX lock_key 1 // 业务逻辑 DO THINGS // 释放锁 DEL lock_key
However, there is a big problem in the above implementation. When client 1 gets the lock, if the following scenario occurs, a deadlock will occur.
The program handles business logic exceptions and fails to release the lock in time. The process hangs and has no chance to release the lock.
The above situation will cause the client that has obtained the lock to occupy the lock forever, and other clients will never be able to obtain it. to the lock.
In order to solve the above deadlock problem, the easiest solution to think of is to set a lock for the lock when applying for a lock and implementing it in Redis. Expiration time, assuming that the time to operate the shared resource will not exceed 10 seconds, then when locking, just set the expiration time of 10 seconds for this key.
But there are still problems with the above operations. There are two commands to lock and set the expiration time. It is possible that only the first one is executed, but the second one fails to execute, for example:
1. SETNX was executed successfully, but EXPIRE failed due to network problems.
2. SETNX was executed successfully, but Redis crashed abnormally, and EXPIRE had no chance to execute.
3. SETNX was executed successfully, and the customer The terminal crashed abnormally, and EXPIRE had no chance to execute
In short, if these two commands cannot be guaranteed to be atomic operations, there is a potential risk that the expiration time setting will fail, and deadlock problems may still occur. . Fortunately, after Redis 2.6.12, Redis has expanded the parameters of the SET command. You can specify the EXPIRE time at the same time as SET. This operation is atomic. For example, the following command sets the lock expiration time to 10 seconds.
SET lock_key 1 EX 10 NXSo far, the deadlock problem has been solved, but there are still other problems. Imagine the following scenario:
客户端1操作共享资源耗时太久，超过了锁的过期时间，锁失效（锁被自动释放）
客户端2加锁成功，开始操作共享资源
客户端1操作共享资源完成，在finally块中手动释放锁，但此时它释放的是客户端2的锁。
这里存在两个严重的问题：
锁过期
释放了别人的锁
第1个问题是评估操作共享资源的时间不准确导致的，如果只是一味增大过期时间，只能缓解问题降低出现问题的概率，依旧无法彻底解决问题。原因在于客户端在拿到锁之后，在操作共享资源时，遇到的场景是很复杂的，既然是预估的时间，也只能是大致的计算，不可能覆盖所有导致耗时变长的场景。
第二个问题在于解锁操作是不够严谨的，因为它是一种不加区分地释放锁的操作，没有对锁的所有权进行检查。如何解决呢？
解决办法是，客户端在加锁时，设置一个只有自己知道的唯一标识进去，例如可以是自己的线程ID，如果是redis实现，就是SET key unique_value EX 10 NX。之后在释放锁时，要先判断这把锁是否归自己持有，只有是自己的才能释放它。
//释放锁 比较unique_value是否相等，避免误释放
if redis.get("key") == unique_value then
return redis.del("key")这里释放锁使用的是GET + DEL两条命令，这时又会遇到原子性问题了。
客户端1执行GET，判断锁是自己的
客户端2执行了SET命令，强制获取到锁（虽然发生概念很低，但要严谨考虑锁的安全性）
客户端1执行DEL，却释放了客户端2的锁
由此可见，以上GET + DEL两个命令还是必须原子的执行才行。怎样原子执行两条命令呢？答案是Lua脚本，可以把以上逻辑写成Lua脚本，让Redis执行。因为Redis处理每个请求是单线程执行的，在执行一个Lua脚本时其它请求必须等待，直到这个Lua脚本处理完成，这样一来GET+DEL之间就不会有其他命令执行了。
以下是使用Lua脚本（unlock.script）实现的释放锁操作的伪代码，其中，KEYS[1]表示lock_key，ARGV[1]是当前客户端的唯一标识，这两个值都是我们在执行 Lua脚本时作为参数传入的。
//Lua脚本语言，释放锁 比较unique_value是否相等，避免误释放
if redis.call("get",KEYS[1]) == ARGV[1] then
return redis.call("del",KEYS[1])
else
return 0
end最后我们执行以下命令，即可
redis-cli --eval unlock.script lock_key , unique_value
这样一路优先下来，整个加锁、解锁流程就更严谨了，先小结一下，基于Redis实现的分布式锁，一个严谨的流程如下：
加锁时要设置过期时间SET lock_key unique_value EX expire_time NX
操作共享资源
释放锁：Lua脚本，先GET判断锁是否归属自己，再DEL释放锁
有了这个严谨的锁模型，我们还需要重新思考之前的那个问题，锁的过期时间不好评估怎么办。
前面提到过，过期时间如果评估得不好，这个锁就会有提前过期的风险，一种妥协的解决方案是，尽量冗余过期时间，降低锁提前过期的概率，但这个方案并不能完美解决问题。是否可以设置这样的方案，加锁时，先设置一个预估的过期时间，然后开启一个守护线程，定时去检测这个锁的失效时间，如果锁快要过期了，操作共享资源还未完成，那么就自动对锁进行续期，重新设置过期时间。
Redisson是一个已封装好这些工作的库，可以说是一种非常优秀的解决方案。Redisson是一个Java语言实现的Redis SDK客户端，在使用分布式锁时，它就采用了自动续期的方案来避免锁过期，这个守护线程我们一般叫它看门狗线程。这个SDK提供的API非常友好，它可以像操作本地锁一样操作分布式锁。客户端一旦加锁成功，就会启动一个watch dog看门狗线程，它是一个后台线程，会每隔一段时间（这段时间的长度与设置的锁的过期时间有关）检查一下，如果检查时客户端还持有锁key（也就是说还在操作共享资源），那么就会延长锁key的生存时间。
那如果客户端在加锁成功后就宕机了呢？宕机了那么看门狗任务就不存在了，也就无法为锁续期了，锁到期自动失效。
上面讨论的情况，都是锁在单个Redis 实例中可能产生的问题，并没有涉及到Redis的部署架构细节。
Redis发展到现在，几种常见的部署架构有：
Single mode;
Master-slave mode;
Sentinel mode;
Cluster mode;
When we use Redis,generally deploy it in the master-slave cluster sentinel mode. The role of the sentinel is to monitor the redis node. Operating status. In the ordinary master-slave mode, when the master crashes, you need to manually switch to make the slave the master. The advantage of using the master-slave sentinel combination is that when the master crashes abnormally, the sentinel can implement automatic failover and promote the slave to the new master. Availability is guaranteed by continuing to provide services. So when the master-slave switch occurs, is the distributed lock still safe?
Imagine this scenario:
Client 1 executes the SET command on the master and the lock is successful
At this time, the master is down abnormally, and the SET command has not yet been synchronized to the slave (master-slave replication is asynchronous)
The sentinel promotes the slave to a new one master, but the lock was lost on the new master, causing client 2 to successfully lock , distributed locks may still be affected. Even if Redis ensures high availability through sentinel, if the master node switches master-slave for some reason, the lock will be lost.
In order to avoid the problem of lock failure caused by Redis instance failure, Redis developer Antirez proposed distribution Formula lock algorithm Redlock. The basic idea of the Redlock algorithm is to let the client and multiple independent Redis instances request locks in sequence. If the client can successfully complete the locking operation with more than half of the instances, then we consider that the client has successfully The distributed lock is obtained, otherwise the lock fails. In this way, even if a single Redis instance fails, because the lock variables are also saved on other instances, the client can still perform lock operations normally and the lock variables will not be lost.
Let’s take a closer look at the execution steps of the Redlock algorithm. The implementation of the Redlock algorithm requires Redis to adopt cluster deployment mode, without sentinel nodes, and N independent Redis instances (officially recommended at least 5 instances). Next, we can complete the locking operation in 3 steps.The first step is for the client to obtain the current time.
The second step is for the client to perform locking operations on N Redis instances in sequence. 
The client can only consider the lock to be successful when two conditions are met. Condition 1 is that the client has successfully obtained the lock from more than half (greater than or equal to N/2 1) of the Redis instances. ;The second condition is that the total time spent by the client in acquiring the lock does not exceed the effective time of the lock.
Why can the operation be considered successful only when most instances are successfully locked? In fact, multiple Redis instances are used together to form a distributed system. There will always be abnormal nodes in a distributed system, so when talking about a distributed system, you need to consider how many abnormal nodes there are without affecting the correct operation of the entire system. This is a fault tolerance problem in a distributed system. The conclusion of this problem is: if there are only faulty nodes, as long as most nodes are normal, the entire system can still provide correct services.After meeting these two conditions, we need torecalculate the effective time of the lock. The result of the calculation is the initial effective time of the lock minus the total time spent by the client to obtain the lock. If the lock's validity time is too late to complete the shared data operation, we can release the lock to avoid the situation where the lock expires before the shared resource operation is completed
当然，如果客户端在和所有实例执行完加锁操作后，没能同时满足这两个条件，那么，客户端就要向所有Redis节点发起释放锁的操作。为什么释放锁，要操作所有的节点呢，不能只操作那些加锁成功的节点吗？因为在某一个Redis节点加锁时，可能因为网络原因导致加锁失败，例如一个客户端在一个Redis实例上加锁成功，但在读取响应结果时由于网络问题导致读取失败，那这把锁其实已经在Redis上加锁成功了。所以释放锁时，不管之前有没有加锁成功，需要释放所有节点上的锁以保证清理节点上的残留的锁。
在Redlock算法中，释放锁的操作和在单实例上释放锁的操作一样，只要执行释放锁的 Lua脚本就可以了。如果N个Redis实例中超过一半的实例正常工作，就能确保分布式锁正常运作。为了提高分布式锁的可靠性，您可以在实际业务应用中使用Redlock算法。
<!-- springboot整合redis -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>package com.example.redisdemo.config;
import com.fasterxml.jackson.annotation.JsonAutoDetect;
import com.fasterxml.jackson.annotation.PropertyAccessor;
import com.fasterxml.jackson.databind.DeserializationFeature;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.data.redis.serializer.Jackson2JsonRedisSerializer;
import org.springframework.data.redis.serializer.RedisSerializer;
import org.springframework.data.redis.serializer.StringRedisSerializer;
/**
* @description: Redis配置类
* @author Keson
* @date 21:20 2022/11/14
* @Param
* @return
* @version 1.0
*/
@Configuration
public class RedisConfig {
@Bean
public RedisTemplate<String, Object> redisTemplate(LettuceConnectionFactory lettuceConnectionFactory) {
// 设置序列化
Jackson2JsonRedisSerializer<Object> jackson2JsonRedisSerializer = new Jackson2JsonRedisSerializer<Object>(Object.class);
ObjectMapper om = new ObjectMapper();
om.setVisibility(PropertyAccessor.ALL, JsonAutoDetect.Visibility.ANY);
om.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
jackson2JsonRedisSerializer.setObjectMapper(om);
// 配置redisTemplate
RedisTemplate<String, Object> redisTemplate = new RedisTemplate<String, Object>();
redisTemplate.setConnectionFactory(lettuceConnectionFactory);
RedisSerializer<?> stringSerializer = new StringRedisSerializer();
redisTemplate.setKeySerializer(stringSerializer);// key序列化
redisTemplate.setValueSerializer(jackson2JsonRedisSerializer);// value序列化
redisTemplate.setHashKeySerializer(stringSerializer);// Hash key序列化
redisTemplate.setHashValueSerializer(jackson2JsonRedisSerializer);// Hash value序列化
redisTemplate.afterPropertiesSet();
return redisTemplate;
}
}package com.example.redisdemo.service;
import com.example.redisdemo.entity.CustomerBalance;
import java.util.concurrent.Callable;
/**
* @author Keson
* @version 1.0
* @description: TODO
* @date 2022/11/14 15:12
*/
public interface RedisService {
<T> T callWithLock(CustomerBalance customerBalance, Callable<T> callable) throws Exception;
}package com.example.redisdemo.service.impl;
import com.example.redisdemo.entity.CustomerBalance;
import com.example.redisdemo.service.RedisService;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.data.redis.connection.RedisStringCommands;
import org.springframework.data.redis.connection.ReturnType;
import org.springframework.data.redis.core.RedisCallback;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.data.redis.core.types.Expiration;
import org.springframework.stereotype.Service;
import java.nio.charset.StandardCharsets;
import java.util.UUID;
import java.util.concurrent.Callable;
import java.util.concurrent.TimeUnit;
/**
* @author Keson
* @version 1.0
* @description: TODO Redis实现分布式锁
* @date 2022/11/14 15:13
*/
@Service
@Slf4j
public class RedisServiceImpl implements RedisService {
//设置默认过期时间
private final static int DEFAULT_LOCK_EXPIRY_TIME = 20;
//自定义lock key前缀
private final static String LOCK_PREFIX = "LOCK:CUSTOMER_BALANCE";
@Autowired
private RedisTemplate redisTemplate;
@Override
public <T> T callWithLock(CustomerBalance customerBalance, Callable<T> callable) throws Exception{
//自定义lock key
String lockKey = getLockKey(customerBalance.getCustomerNumber(), customerBalance.getSubAccountNumber(), customerBalance.getCurrencyCode());
//将UUID当做value，确保唯一性
String lockReference = UUID.randomUUID().toString();
try {
if (!lock(lockKey, lockReference, DEFAULT_LOCK_EXPIRY_TIME, TimeUnit.SECONDS)) {
throw new Exception("lock加锁失败");
}
return callable.call();
} finally {
unlock(lockKey, lockReference);
}
}
//定义lock key
String getLockKey(String customerNumber, String subAccountNumber, String currencyCode) {
return String.format("%s:%s:%s:%s", LOCK_PREFIX, customerNumber, subAccountNumber, currencyCode);
}
//redis加锁
private boolean lock(String key, String value, long timeout, TimeUnit timeUnit) {
Boolean locked;
try {
//SET_IF_ABSENT --> NX: Only set the key if it does not already exist.
//SET_IF_PRESENT --> XX: Only set the key if it already exist.
locked = (Boolean) redisTemplate.execute((RedisCallback<Boolean>) connection ->
connection.set(key.getBytes(StandardCharsets.UTF_8), value.getBytes(StandardCharsets.UTF_8),
Expiration.from(timeout, timeUnit), RedisStringCommands.SetOption.SET_IF_ABSENT));
} catch (Exception e) {
log.error("Lock failed for redis key: {}, value: {}", key, value);
locked = false;
}
return locked != null && locked;
}
//redis解锁
private boolean unlock(String key, String value) {
try {
//使用lua脚本保证删除的原子性，确保解锁
String script = "if redis.call('get', KEYS[1]) == ARGV[1] " +
"then return redis.call('del', KEYS[1]) " +
"else return 0 end";
Boolean unlockState = (Boolean) redisTemplate.execute((RedisCallback<Boolean>) connection ->
connection.eval(script.getBytes(), ReturnType.BOOLEAN, 1,
key.getBytes(StandardCharsets.UTF_8), value.getBytes(StandardCharsets.UTF_8)));
return unlockState == null || !unlockState;
} catch (Exception e) {
log.error("unLock failed for redis key: {}, value: {}", key, value);
return false;
}
}
} @Override
public int updateById(CustomerBalance customerBalance) throws Exception {
return redisService.callWithLock(customerBalance, ()-> customerBalanceMapper.updateById(customerBalance));
}The above is the detailed content of How to use Redis to implement distributed locks in SpringBoot. For more information, please follow other related articles on the PHP Chinese website!
SpringBoot project building steps
Commonly used database software
What are the in-memory databases?
Which one has faster reading speed, mongodb or redis?
How to use redis as a cache server
How redis solves data consistency
How do mysql and redis ensure double-write consistency?
What data does redis cache generally store?
![[Web front-end] Node.js quick start](https://img.php.cn/upload/course/000/000/067/662b5d34ba7c0227.png)
