How to use Java GenericObjectPool object pooling technology

Java BasePooledObjectFactory Object Pooling Technology

Usually when the creation and destruction of an object is very time-consuming, we will not create and destroy it frequently, but consider reuse. One way to reuse objects is object pooling. Put the created objects into the pool and maintain them. The next time you use them, you can directly use the objects that have been created in the pool and continue to use them. This is the idea of ​​pooling.

Apache Commons Pool is an object pool framework that provides a complete set of APIs for implementing object pooling. It provides three types of object pools: GenericKeyedObjectPool, SoftReferenceObjectPool and GenericObjectPool. GenericObjectPool is our most commonly used object pool, and its internal implementation is also the most complex.

GenericObjectPool

GenericObjectPool is a general object pool framework with which we can implement a robust object pool. The UML diagram is as follows:

GenericObjectPool implements the ObjectPool interface, and ObjectPool is the core interface of the object pool, which defines the behavior that an object pool should implement.

public interface ObjectPool<T> extends Closeable {
    /**
     * 从池中借走到一个对象
     */
    T borrowObject() throws Exception, NoSuchElementException, IllegalStateException;
    /**
     * 把对象归还给对象池
     */
    void returnObject(T var1) throws Exception;
    /**
     * 验证对象的有效性
     */
    void invalidateObject(T var1) throws Exception;

    /**
     * 往池中添加一个对象
     */
    void addObject() throws Exception, IllegalStateException, UnsupportedOperationException;
    /**
     * 返回对象池中有多少对象是空闲的，也就是能够被借走的对象的数量。
     */
    int getNumIdle();
    /**
     * 返回对象池中有对象对象是活跃的，也就是已经被借走的，在使用中的对象的数量。
     */
    int getNumActive();
    /**
     * 清理对象池。注意是清理不是清空，该方法要求的是，清理所有空闲对象，释放相关资源。
     */
    void clear() throws Exception, UnsupportedOperationException;
    /**
     * 关闭对象池。这个方法可以达到清空的效果，清理所有对象以及相关资源。
     */
    void close();
}

BasePooledObjectFactory

Java BasePooledObjectFactory object pooling technology

To use GenericObjectPool you only need to create an object factory class and inherit BasePooledObjectFactory And override its create() and destroyObject().
As in the following: SftpPool.java

public interface PooledObjectFactory<T> {
    /**
     * 创建一个可由池提供服务的实例，并将其封装在由池管理的PooledObject中。
     */
    PooledObject<T> makeObject() throws Exception;

    /**
     *  销毁池不再需要的实例
     */
    void destroyObject(PooledObject<T> var1) throws Exception;

    /**
     * 确保实例可以安全地由池返回
     */
    boolean validateObject(PooledObject<T> var1);

    /**
     * 重新初始化池返回的实例
     */
    void activateObject(PooledObject<T> var1) throws Exception;

    /**
     * 取消初始化要返回到空闲对象池的实例
     */
    void passivateObject(PooledObject<T> var1) throws Exception;
}

Configuration class GenericObjectPoolConfig

GenericObjectPoolConfig is the encapsulation GenericObjectPool configuration A simple "struct", this class is not thread-safe; it is only used to provide properties used when creating the pool. In most cases, you can use the default parameters provided by GenericObjectPoolConfig to meet daily needs.

Working principle process

• ##Construction method

  When we execute the construction method, the main job is to create a LinkedList type container that stores objects, which is the conceptual meaning "Pool" on
  

• Get objects from the object pool

  Get the objects in the pool through the borrowObject() command. The source code is more complicated. Simply put, it is to get it from LinkedList If an object does not exist, you must call the makeObject() method of the first parameter Factory class in the constructor to create an object and then obtain it. After obtaining the object, you must call the validateObject method to determine whether the object is available. If so, Use only what is available. LinkedList container minus one
  

• Return the object to the thread pool

  To put it simply, call the validateObject method to determine whether the object is available. If it is available, return it to the pool. LinkedList container Add one, if it is not possible, call the destroyObject method to destroy
  

The above three steps are the simplest process, because the process steps of taking and returning are fixed in the borrowObject and returnObject methods , so we only need to rewrite the makeObject(), validateObject and destroyObject methods of the Factory class to achieve the simplest pool management control. By passing in the Factory class object through the constructor method, we can create the simplest object pool management class. . This is a better decoupling design pattern. The borrowing and returning process is as shown in the figure below:

Use Demo

<dependency>
    <groupId>org.apache.commons</groupId>
    <artifactId>commons-pool2</artifactId>
    <version>2.7.0</version>
</dependency>

<!-- https://mvnrepository.com/artifact/com.jcraft/jsch -->
<dependency>
    <groupId>com.jcraft</groupId>
    <artifactId>jsch</artifactId>
    <version>0.1.55</version>
</dependency>

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <artifactId>vipsoft-parent</artifactId>
        <groupId>com.vipsoft.boot</groupId>
        <version>1.0-SNAPSHOT</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>

    <artifactId>vipsoft-sftp</artifactId>
    <version>1.0-SNAPSHOT</version>

    <dependencies>

        <dependency>
            <groupId>org.apache.commons</groupId>
            <artifactId>commons-pool2</artifactId>
            <version>2.7.0</version>
        </dependency>

        <!-- https://mvnrepository.com/artifact/com.jcraft/jsch -->
        <dependency>
            <groupId>com.jcraft</groupId>
            <artifactId>jsch</artifactId>
            <version>0.1.55</version>
        </dependency>

        <dependency>
            <groupId>org.eclipse.paho</groupId>
            <artifactId>org.eclipse.paho.client.mqttv3</artifactId>
            <version>1.2.5</version>
        </dependency>


        <dependency>
            <groupId>cn.hutool</groupId>
            <artifactId>hutool-all</artifactId>
            <version>5.3.6</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
            <exclusions>
                <exclusion>
                    <groupId>org.junit.vintage</groupId>
                    <artifactId>junit-vintage-engine</artifactId>
                </exclusion>
            </exclusions>
        </dependency>

    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>
</project>

application.yaml

server:
  port: 8088
  application:
    name: sftp Demo


sftp:
  host: 172.16.3.88 # 服务器ip
  port: 22 # ssh端口
  username: root # 用户名
  password: root # 密码
  # 连接池参数
  pool:
    max-total: 10
    max-idle: 10
    min-idle: 5

SftpPoolException.java

package com.vipsoft.sftp.exception;


/**
 * sftp连接池异常
 */
public class SftpPoolException extends RuntimeException {

    private static final long serialVersionUID = 1L;

    /**
     * Constructs a new runtime exception with {@code null} as its
     * detail message.  The cause is not initialized, and may subsequently be
     * initialized by a call to {@link #initCause}.
     */
    public SftpPoolException() {
    }

    /**
     * Constructs a new runtime exception with the specified detail message.
     * The cause is not initialized, and may subsequently be initialized by a
     * call to {@link #initCause}.
     *
     * @param message the detail message. The detail message is saved for
     *                later retrieval by the {@link #getMessage()} method.
     */
    public SftpPoolException(String message) {
        super(message);
    }

    /**
     * Constructs a new runtime exception with the specified detail message and
     * cause.  <p>Note that the detail message associated with
     * {@code cause} is <i>not</i> automatically incorporated in
     * this runtime exception&#39;s detail message.
     *
     * @param message the detail message (which is saved for later retrieval
     *                by the {@link #getMessage()} method).
     * @param cause   the cause (which is saved for later retrieval by the
     *                {@link #getCause()} method).  (A <tt>null</tt> value is
     *                permitted, and indicates that the cause is nonexistent or
     *                unknown.)
     * @since 1.4
     */
    public SftpPoolException(String message, Throwable cause) {
        super(message, cause);
    }

    /**
     * Constructs a new runtime exception with the specified cause and a
     * detail message of <tt>(cause==null ? null : cause.toString())</tt>
     * (which typically contains the class and detail message of
     * <tt>cause</tt>).  This constructor is useful for runtime exceptions
     * that are little more than wrappers for other throwables.
     *
     * @param cause the cause (which is saved for later retrieval by the
     *              {@link #getCause()} method).  (A <tt>null</tt> value is
     *              permitted, and indicates that the cause is nonexistent or
     *              unknown.)
     * @since 1.4
     */
    public SftpPoolException(Throwable cause) {
        super(cause);
    }

    /**
     * Constructs a new runtime exception with the specified detail
     * message, cause, suppression enabled or disabled, and writable
     * stack trace enabled or disabled.
     *
     * @param message            the detail message.
     * @param cause              the cause.  (A {@code null} value is permitted,
     *                           and indicates that the cause is nonexistent or unknown.)
     * @param enableSuppression  whether or not suppression is enabled
     *                           or disabled
     * @param writableStackTrace whether or not the stack trace should
     *                           be writable
     * @since 1.7
     */
    public SftpPoolException(String message, Throwable cause, boolean enableSuppression, boolean writableStackTrace) {
        super(message, cause, enableSuppression, writableStackTrace);
    }

}

config

SftpConfig.java

package com.vipsoft.sftp.config;

import com.vipsoft.sftp.pool.SftpFactory;
import com.vipsoft.sftp.pool.SftpPool;
import com.vipsoft.sftp.utils.SftpUtil;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
@EnableConfigurationProperties(SftpProperties.class)
public class SftpConfig {
    // 工厂
    @Bean
    public SftpFactory sftpFactory(SftpProperties properties) {
        return new SftpFactory(properties);
    }

    // 连接池
    @Bean
    public SftpPool sftpPool(SftpFactory sftpFactory) {
        return new SftpPool(sftpFactory);
    }

    // 辅助类
    @Bean
    public SftpUtil sftpUtil(SftpPool sftpPool) {
        return new SftpUtil(sftpPool);
    }
}

SftpProperties.java

package com.vipsoft.sftp.config;

import com.jcraft.jsch.ChannelSftp;
import org.apache.commons.pool2.impl.GenericObjectPoolConfig;
import org.springframework.boot.context.properties.ConfigurationProperties;

@ConfigurationProperties(prefix = "sftp")
public class SftpProperties {

    private String host;
    private int port = 22;
    private String username = "root";
    private String password = "root";
    private Pool pool = new Pool();

    public String getHost() {
        return host;
    }

    public void setHost(String host) {
        this.host = host;
    }

    public int getPort() {
        return port;
    }

    public void setPort(int port) {
        this.port = port;
    }

    public String getUsername() {
        return username;
    }

    public void setUsername(String username) {
        this.username = username;
    }

    public String getPassword() {
        return password;
    }

    public void setPassword(String password) {
        this.password = password;
    }

    public Pool getPool() {
        return pool;
    }

    public void setPool(Pool pool) {
        this.pool = pool;
    }

    public static class Pool extends GenericObjectPoolConfig<ChannelSftp> {

        private int maxTotal = DEFAULT_MAX_TOTAL;
        private int maxIdle = DEFAULT_MAX_IDLE;
        private int minIdle = DEFAULT_MIN_IDLE;

        public Pool() {
            super();
        }
        @Override
        public int getMaxTotal() {
            return maxTotal;
        }
        @Override
        public void setMaxTotal(int maxTotal) {
            this.maxTotal = maxTotal;
        }
        @Override
        public int getMaxIdle() {
            return maxIdle;
        }
        @Override
        public void setMaxIdle(int maxIdle) {
            this.maxIdle = maxIdle;
        }
        @Override
        public int getMinIdle() {
            return minIdle;
        }
        @Override
        public void setMinIdle(int minIdle) {
            this.minIdle = minIdle;
        }
    }
}

Pool

SftpFactory. java

package com.vipsoft.sftp.pool;

import com.jcraft.jsch.ChannelSftp;
import com.jcraft.jsch.JSch;
import com.jcraft.jsch.JSchException;
import com.jcraft.jsch.Session;
import com.vipsoft.sftp.config.SftpProperties;
import com.vipsoft.sftp.exception.SftpPoolException;
import org.apache.commons.pool2.BasePooledObjectFactory;
import org.apache.commons.pool2.PooledObject;
import org.apache.commons.pool2.impl.DefaultPooledObject;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.util.Properties;

public class SftpFactory extends BasePooledObjectFactory<ChannelSftp> {

    private  final Logger logger = LoggerFactory.getLogger(this.getClass());

    private SftpProperties properties;

    public SftpProperties getProperties() {
        return properties;
    }

    public void setProperties(SftpProperties properties) {
        this.properties = properties;
    }

    public SftpFactory(SftpProperties properties) {
        this.properties = properties;
    }

    @Override
    public ChannelSftp create() {
        try {
            JSch jsch = new JSch();
            Session sshSession = jsch.getSession(properties.getUsername(), properties.getHost(), properties.getPort());
            sshSession.setPassword(properties.getPassword());
            Properties sshConfig = new Properties();
            sshConfig.put("StrictHostKeyChecking", "no");
            sshSession.setConfig(sshConfig);
            sshSession.connect();
            ChannelSftp channel = (ChannelSftp) sshSession.openChannel("sftp");
            channel.connect();
            return channel;
        } catch (JSchException e) {
            throw new SftpPoolException("连接sfpt失败", e);
        }
    }

    @Override
    public PooledObject<ChannelSftp> wrap(ChannelSftp channelSftp) {
        return new DefaultPooledObject<>(channelSftp);
    }

    // 销毁对象
    @Override
    public void destroyObject(PooledObject<ChannelSftp> p) {
        ChannelSftp channelSftp = p.getObject();
        channelSftp.disconnect();
    }
}

SftpPool.java

package com.vipsoft.sftp.pool;

import com.jcraft.jsch.ChannelSftp;
import org.apache.commons.pool2.impl.GenericObjectPool;

public class SftpPool<T> extends GenericObjectPool<ChannelSftp> {

    public SftpPool(SftpFactory factory) {
        super(factory,factory.getProperties().getPool());
    }

    /**
     * 获取一个sftp连接对象
     * @return sftp连接对象
     */
    @Override
    public ChannelSftp borrowObject() throws Exception {
        return super.borrowObject();
    }

    /**
     * 归还一个sftp连接对象
     * @param channelSftp sftp连接对象
     */
    @Override
    public void returnObject(ChannelSftp channelSftp) {
        if (channelSftp!=null) {
            super.returnObject(channelSftp);
        }
    }

}

Utils

ByteUtil.java

package com.vipsoft.sftp.utils;

import com.jcraft.jsch.ChannelSftp;
import com.jcraft.jsch.SftpException;
import com.vipsoft.sftp.exception.SftpPoolException;
import com.vipsoft.sftp.pool.SftpPool;

import java.io.InputStream;

public class SftpUtil {

    private SftpPool pool;

    public SftpUtil(SftpPool pool) {
        this.pool = pool;
    }

    /**
     * 下载文件
     *
     * @param dir  远程目录
     * @param name 远程文件名
     * @return 文件字节数组
     */
    public byte[] download(String dir, String name) {
        ChannelSftp sftp = null;
        try {
            sftp = pool.borrowObject();
            sftp.cd(dir);
            InputStream in = sftp.get(name);
            return ByteUtil.inputStreamToByteArray(in);
        } catch (Exception e) {
            throw new SftpPoolException("sftp下载文件出错", e);
        } finally {
            pool.returnObject(sftp);
        }
    }

    /**
     * 上传文件
     *
     * @param dir  远程目录
     * @param name 远程文件名
     * @param in   输入流
     */
    public void upload(String dir, String name, InputStream in) {
        ChannelSftp sftp = null;
        try {
            sftp = pool.borrowObject();
            mkdirs(sftp, dir);
            sftp.cd(dir);
            sftp.put(in, name);
        } catch (Exception e) {
            throw new SftpPoolException("sftp上传文件出错", e);
        } finally {
            pool.returnObject(sftp);
        }
    }

    /**
     * 删除文件
     *
     * @param dir  远程目录
     * @param name 远程文件名
     */
    public void delete(String dir, String name) {
        ChannelSftp sftp = null;
        try {
            sftp = pool.borrowObject();
            sftp.cd(dir);
            sftp.rm(name);
        } catch (Exception e) {
            throw new SftpPoolException("sftp删除文件出错", e);
        } finally {
            pool.returnObject(sftp);
        }
    }

    /**
     * 递归创建多级目录
     *
     * @param dir 多级目录
     */
    private void mkdirs(ChannelSftp sftp, String dir) {
        String[] folders = dir.split("/");
        try {
            sftp.cd("/");
            for (String folder : folders) {
                if (folder.length() > 0) {
                    try {
                        sftp.cd(folder);
                    } catch (Exception e) {
                        sftp.mkdir(folder);
                        sftp.cd(folder);
                    }
                }
            }
        } catch (SftpException e) {
            throw new SftpPoolException("sftp创建目录出错", e);
        }
    }
}

Test

SftpTest.java

package com.vipsoft.sftp;

import com.vipsoft.sftp.utils.SftpUtil;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;

@SpringBootTest
public class SftpTest {

    @Autowired
    private SftpUtil sftpUtil;

    @Test
    void downloadTest() {
        byte[] dockerfiles = sftpUtil.download("/opt/demo/", "Dockerfile");
        System.out.println("FileSize =>" + dockerfiles.length);
    }

}

The above is the detailed content of How to use Java GenericObjectPool object pooling technology. For more information, please follow other related articles on the PHP Chinese website!