How SpringBoot implements repeated request interception based on filters and memory

For some interfaces that request the server, there may be repeated requests. If it is a query operation, it is not a big deal. However, if it involves a write operation, once it is repeated, it may have serious consequences on the business logic, such as transactions. If the interface is requested repeatedly, repeated orders may be placed.

Here we use filters to filter requests entering the server to filter requests from the same client to the same interface.

 @Slf4j
 @Component
 public class IRequestFilter extends OncePerRequestFilter {
     @Resource
     private FastMap fastMap;
 
     @Override
     protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response, FilterChain chain) throws ServletException, IOException {
         ServletRequestAttributes attributes = (ServletRequestAttributes) RequestContextHolder.getRequestAttributes();
         String address = attributes != null ? attributes.getRequest().getRemoteAddr() : UUID.randomUUID().toString();
         if (Objects.equals(request.getMethod(), "GET")) {
             StringBuilder str = new StringBuilder();
             str.append(request.getRequestURI()).append("|")
                     .append(request.getRemotePort()).append("|")
                     .append(request.getLocalName()).append("|")
                     .append(address);
             String hex = DigestUtil.md5Hex(new String(str));
             log.info("请求的MD5值为:{}", hex);
             if (fastMap.containsKey(hex)) {
                 throw new IllegalStateException("请求重复，请稍后重试!");
             }
             fastMap.put(hex, 10 * 1000L);
             fastMap.expired(hex, 10 * 1000L, (key, val) -> System.out.println("map:" + fastMap + ",删除的key:" + key + ",线程名:" + Thread.currentThread().getName()));
         }
         log.info("请求的 address:{}", address);
         chain.doFilter(request, response);
     }
 }

By inheriting the OncePerRequestFilter filter in Spring, it is ensured that only one filter is passed in a request without repeated execution

By obtaining the request body The data is calculated and the MD5 value is stored in the memory-based FastMap. The key of FastMap is the MD5 value. The value indicates how long the request cannot be repeated. What is configured here is that the request cannot be repeated within 10 seconds. By calling the expired() method of FastMap, set the expiration time of the request and the callback function when it expires

 @Component
 public class FastMap {
     /**
      * 按照时间顺序保存了会过期key集合，为了实现快速删除，结构：时间戳 -> key 列表
      */
     private final TreeMap<Long, List<String>> expireKeysMap = new TreeMap<>();
     /**
      * 保存会过期key的过期时间
      */
     private final Map<String, Long> keyExpireMap = new ConcurrentHashMap<>();
     /**
      * 保存键过期的回调函数
      */
     private final HashMap<String, ExpireCallback<String, Long>> keyExpireCallbackMap = new HashMap<>();
     private final ReentrantReadWriteLock readWriteLock = new ReentrantReadWriteLock();
     /**
      * 数据写锁
      */
     private final Lock dataWriteLock = readWriteLock.writeLock();
     /**
      * 数据读锁
      */
     private final Lock dataReadLock = readWriteLock.readLock();
     private final ReentrantReadWriteLock expireKeysReadWriteLock = new ReentrantReadWriteLock();
     /**
      * 过期key写锁
      */
     private final Lock expireKeysWriteLock = expireKeysReadWriteLock.writeLock();
     /**
      * 过期key读锁
      */
     private final Lock expireKeysReadLock = expireKeysReadWriteLock.readLock();
     /**
      * 定时执行服务(全局共享线程池)
      */
     private volatile ScheduledExecutorService scheduledExecutorService;
     /**
      * 100万，1毫秒=100万纳秒
      */
     private static final int ONE_MILLION = 100_0000;
     /**
      * 构造器，enableExpire配置是否启用过期，不启用排序
      */
     public FastMap() {
         this.init();
     }
     /**
      * 初始化
      */
     private void init() {
         // 双重校验构造一个单例的scheduledExecutorService
         if (scheduledExecutorService == null) {
             synchronized (FastMap.class) {
                 if (scheduledExecutorService == null) {
                     // 启用定时器，定时删除过期key,1秒后启动，定时1秒, 因为时间间隔计算基于nanoTime,比timer.schedule更靠谱
                     scheduledExecutorService = new ScheduledThreadPoolExecutor(1, runnable -> {
                         Thread thread = new Thread(runnable, "expireTask-" + UUID.randomUUID());
                         thread.setDaemon(true);
                         return thread;
                     });
                 }
             }
         }
     }
     public boolean containsKey(Object key) {
         dataReadLock.lock();
         try {
             return this.keyExpireMap.containsKey(key);
         } finally {
             dataReadLock.unlock();
         }
     }
     public Long put(String key, Long value) {
         dataWriteLock.lock();
         try {
             return this.keyExpireMap.put(key, value);
         } finally {
             dataWriteLock.unlock();
         }
     }
     public Long remove(Object key) {
         dataWriteLock.lock();
         try {
             return this.keyExpireMap.remove(key);
         } finally {
             dataWriteLock.unlock();
         }
     }
     public Long expired(String key, Long ms, ExpireCallback<String, Long> callback) {
         // 对过期数据写上锁
         expireKeysWriteLock.lock();
         try {
             // 使用nanoTime消除系统时间的影响，转成毫秒存储降低timeKey数量,过期时间精确到毫秒级别
             Long expireTime = (System.nanoTime() / ONE_MILLION + ms);
             this.keyExpireMap.put(key, expireTime);
             List<String> keys = this.expireKeysMap.get(expireTime);
             if (keys == null) {
                 keys = new ArrayList<>();
                 keys.add(key);
                 this.expireKeysMap.put(expireTime, keys);
             } else {
                 keys.add(key);
             }
             if (callback != null) {
                 // 设置的过期回调函数
                 this.keyExpireCallbackMap.put(key, callback);
             }
             // 使用延时服务调用清理key的函数，可以及时调用过期回调函数
             // 同key重复调用，会产生多个延时任务，就是多次调用清理函数，但是不会产生多次回调，因为回调取决于过期时间和回调函数）
             scheduledExecutorService.schedule(this::clearExpireData, ms, TimeUnit.MILLISECONDS);
 
             //假定系统时间不修改前提下的过期时间
             return System.currentTimeMillis() + ms;
         } finally {
             expireKeysWriteLock.unlock();
         }
     }
     /**
      * 清理过期的数据
      * 调用时机：设置了过期回调函数的key的延时任务调用
      */
     private void clearExpireData() {
         // 查找过期key
         Long curTimestamp = System.nanoTime() / ONE_MILLION;
         Map<Long, List<String>> expiredKeysMap = new LinkedHashMap<>();
         expireKeysReadLock.lock();
         try {
             // 过期时间在【从前至此刻】区间内的都为过期的key
             // headMap():获取从头到 curTimestamp 元素的集合：不包含 curTimestamp
             SortedMap<Long, List<String>> sortedMap = this.expireKeysMap.headMap(curTimestamp, true);
             expiredKeysMap.putAll(sortedMap);
         } finally {
             expireKeysReadLock.unlock();
         }
 
         for (Map.Entry<Long, List<String>> entry : expiredKeysMap.entrySet()) {
             for (String key : entry.getValue()) {
                 // 删除数据
                 Long val = this.remove(key);
                 // 首次调用删除（val!=null，前提：val存储值都不为null）
                 if (val != null) {
                     // 如果存在过期回调函数，则执行回调
                     ExpireCallback<String, Long> callback;
                     expireKeysReadLock.lock();
                     try {
                         callback = this.keyExpireCallbackMap.get(key);
                     } finally {
                         expireKeysReadLock.unlock();
                     }
                     if (callback != null) {
                         // 回调函数创建新线程调用，防止因为耗时太久影响线程池的清理工作
                         // 这里为什么不用线程池调用，因为ScheduledThreadPoolExecutor线程池仅支持核心线程数设置，不支持非核心线程的添加
                         // 核心线程数用一个就可以完成清理工作，添加额外的核心线程数浪费了
                         new Thread(() -> callback.onExpire(key, val), "callback-thread-" + UUID.randomUUID()).start();
                     }
                 }
                 this.keyExpireCallbackMap.remove(key);
             }
             this.expireKeysMap.remove(entry.getKey());
         }
     }
 }

FastMap implements scheduled thread tasks through the ScheduledExecutorService interface Automatic deletion of requests at expiration time.

The above is the detailed content of How SpringBoot implements repeated request interception based on filters and memory. For more information, please follow other related articles on the PHP Chinese website!