How to implement log file monitoring and read related data in Java

Project Requirements

Because the data center project team needs to implement the need to monitor folders or log files and read dirty data in the corresponding format, so that corresponding business can be performed when files and folders change. Process; so the implementation of relevant business and technology selection are recorded here.

Apache Commons-IO

First you need to add the corresponding dependencies:

<dependency>
 <groupId>commons-io</groupId>
 <artifactId>commons-io</artifactId>
 <version>2.7</version>
</dependency>

You can choose the version by yourself. It should be noted here that version 2.7 and above require Java 8

Official website screenshot:

##org.apache.commons.io.monitor package in Apache Commos IO provides files System monitoring function.

##Core knowledge

Summary in one sentence: Through the observer pattern and event listening mechanism,
• FileAlterationObserver

  As the core, through FileAlterationListener, FileAlterationMonitor, you can realize event monitoring of the file system.

Overall process:

Customize the file listening class and inherit
• FileAlterationListenerAdaptor

  to implement file monitoring Handling directory creation, modification, and deletion events

Customize the file monitoring class and create an observer by specifying the directory
• FileAlterationObserver

Add a file system observer to this monitor and add a file listener
Call via
• ApplicationRunner

  or CommandLineRunner and Execute

code to implement

File monitoring class:

• /**
   * 文件监听类
   * @author Greenarrow
   * @date 2022-05-12 14:43
   **/
  public class FileListener extends FileAlterationListenerAdaptor {
  
      private static Logger logger = LoggerFactory.getLogger(FileListener.class);
      @Override
      public void onStart(FileAlterationObserver observer) {
          super.onStart(observer);
          logger.info("onStart");
      }
  
      @Override
      public void onDirectoryCreate(File directory) {
          logger.info("[新建]:" + directory.getAbsolutePath());
      }
  
      @Override
      public void onDirectoryChange(File directory) {
          logger.info("[修改]:" + directory.getAbsolutePath());
      }
  
      @Override
      public void onDirectoryDelete(File directory) {
          logger.info("[删除]:" + directory.getAbsolutePath());
      }
  
      @Override
      public void onFileCreate(File file) {
          String compressedPath = file.getAbsolutePath();
          logger.info("[新建]:" + compressedPath);
          List<String> contentList = null;
          try {
              if (file.canRead()){
                  // 将文件按行读取为字符串集合
                  contentList = FileUtils.readLines(new File(compressedPath), StandardCharsets.UTF_8);
                  if (CollectionUtil.isNotEmpty(contentList)){
                      // 获取对应格式的数据并输出，这里可自行添加业务处理
                      List<String> dirtyRecord  = contentList.stream().filter(s -> s.startsWith("{") && s.endsWith("}")).collect(Collectors.toList());
                      dirtyRecord.forEach(System.out::println);
                  }
              }
  
          } catch (IOException e) {
              e.printStackTrace();
              logger.error("读取文件内容失败",e);
          }
      }
  
      @Override
      public void onFileChange(File file) {
          String compressedPath = file.getAbsolutePath();
          logger.info("[修改]:" + compressedPath);
      }
  
      @Override
      public void onFileDelete(File file) {
          logger.info("[删除]:" + file.getAbsolutePath());
      }
  
      @Override
      public void onStop(FileAlterationObserver observer) {
          super.onStop(observer);
          logger.info("onStop");
      }
  }

File monitoring class:

• /**
   * 文件监听测试 demo
   * @author Greenarrow
   * @date 2022-05-12 14:45
   **/
  public class FileMonitor {
  
      FileAlterationMonitor monitor = null;
  
      public FileMonitor(long interval) throws Exception {
          monitor = new FileAlterationMonitor(interval);
      }
  
      /**
       * 给文件添加监听
       * @param path
       * @param listener
       */
      public void monitor(String path, FileAlterationListener listener) {
          FileAlterationObserver observer = new FileAlterationObserver(new File(path));
          monitor.addObserver(observer);
          observer.addListener(listener);
      }
  
      public void stop() throws Exception {
          monitor.stop();
      }
  
      public void start() throws Exception {
          monitor.start();
  
      }
  }

Customize Runner and implement
• CommandLineRunner

  :

  /**
   * 项目启动之后开启文件监听功能
   * @author Greenarrow
   * @date 2022-05-12 10:02
   **/
  @Component
  // @Order(Integer.MIN_VALUE)
  public class DirtyRecordRunner implements CommandLineRunner {
  
      private static Logger logger = LoggerFactory.getLogger(DirtyRecordRunner.class);
  
      @Value("${test}")
      private String path;
  
      @Override
      public void run(String... args) throws Exception {
          logger.info(this.getClass().getName()+"[开启文件夹监听功能]");
          FileMonitor fileMonitor = new FileMonitor(1000);
          fileMonitor.monitor(path,new FileListener());
          fileMonitor.start();
      }
  }

  The above is the detailed content of How to implement log file monitoring and read related data in Java. For more information, please follow other related articles on the PHP Chinese website!