Home > Java > Spring SFTP Intergation: Handling moving files to another directory

Spring SFTP Intergation: Handling moving files to another directory

WBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWB
Release: 2024-02-09 14:18:09
forward
825 people have browsed it

php editor Xiaoxin introduces Spring SFTP Integration is a powerful framework that can help developers handle the task of moving files from one directory to another. Whether on a local server or a remote server, this framework makes it easy to move and manage files. It provides a series of APIs and configuration options that allow developers to customize the logic of moving files and handle various exceptions. Whether in enterprise-level applications or personal projects, Spring SFTP Integration is a very practical tool worth exploring.

Question content

I am new to spring intergation. I need to process files from sftp server from user1/upload directory and then move them to user1/processed directory. My code generally works fine, but I have two problems:

  1. When I restart the application, the directory user1/processed is deleted along with all the files that existed before. I just want to write more files there without clearing the directory every time.

  2. Every time I start the application I receive files (as I see their names and I print to the console) I receive old files which have been moved to processed in the directory. This seems really weird because I don't see these files when I connect to sftp via other tools like winscp. Has the old file list been honored somewhere?

@Value("${cielo.sftp.host}")
    private String sftpHost;
    @Value("${cielo.sftp.port}")
    private int sftpPort;
    @Value("${cielo.sftp.user}")
    private String sftpUser;
    @Value("${cielo.sftp.pass}")
    private String sftpPasword;
    @Value("${cielo.sftp.remotedir}")
    private String sftpRemoteDirectoryDownload;

    @Value("${cielo.sftp.localdir}")
    private String sftpLocalDirectoryDownload;
    @Value("${cielo.sftp.filter}")
    private String sftpRemoteDirectoryDownloadFilter;

    @Bean
    @Order(Ordered.HIGHEST_PRECEDENCE)
    public SessionFactory<LsEntry> sftpSessionFactory() {
        DefaultSftpSessionFactory factory = new DefaultSftpSessionFactory(true);
        factory.setHost(sftpHost);
        factory.setPort(sftpPort);
        factory.setUser(sftpUser);
        factory.setPassword(sftpPasword);

        factory.setAllowUnknownKeys(true); //Set to true to allow connections to hosts with unknown (or changed) keys. Its default is 'false'. If false, a pre-populated knownHosts file is required.
        return new CachingSessionFactory<>(factory);
    }


    @Bean
    @Order(Ordered.HIGHEST_PRECEDENCE - 1)
    public SftpInboundFileSynchronizer sftpInboundFileSynchronizer(final SessionFactory<LsEntry> sftpSessionFactory) {
        SftpInboundFileSynchronizer fileSynchronizer =
                new SftpInboundFileSynchronizer(sftpSessionFactory);
        fileSynchronizer.setDeleteRemoteFiles(false);
        fileSynchronizer.setRemoteDirectory(sftpRemoteDirectoryDownload);
        fileSynchronizer
                .setFilter(new SftpSimplePatternFileListFilter(sftpRemoteDirectoryDownloadFilter)); //todo maybe use RegexPatternFileListFilter?
        return fileSynchronizer;
    }

    @Bean
    @Order(Ordered.HIGHEST_PRECEDENCE - 2)
    @InboundChannelAdapter(channel = "fromSftpChannel", poller = @Poller(fixedDelay = "1000"))
    //@InboundChannelAdapter(channel = "fromSftpChannel", poller = @Poller(cron = "${cielo.sftp.poller.cron}"))

    public MessageSource<File> sftpMessageSource(final SftpInboundFileSynchronizer sftpInboundFileSynchronizer) {
        SftpInboundFileSynchronizingMessageSource source =
                new SftpInboundFileSynchronizingMessageSource(sftpInboundFileSynchronizer);
        source.setLocalDirectory(new File("/tmp/local"));
        source.setAutoCreateLocalDirectory(true);
        source.setLocalFilter(new AcceptOnceFileListFilter<File>());
        return source;
    }

    @Bean
    @ServiceActivator(
            inputChannel = "fromSftpChannel")
    public MessageHandler resultFileHandler() {
        return new MessageHandler() {
            @Override
            public void handleMessage(final Message<?> message) throws MessagingException {
                String payload = String.valueOf(message.getPayload());
                System.err.println(payload);
            }
        };
    }

    private static final SpelExpressionParser PARSER = new SpelExpressionParser();

    @Bean(name="fromSftpChannel")
    public MessageChannel fromSftpChannel() {
        return new PublishSubscribeChannel();
    }

    @Bean
    @ServiceActivator(inputChannel = "fromSftpChannel")
    @Order(Ordered.LOWEST_PRECEDENCE)
    public MessageHandler moveFile() {
        SftpOutboundGateway sftpOutboundGateway = new  SftpOutboundGateway(sftpSessionFactory(), AbstractRemoteFileOutboundGateway.Command.MV.getCommand(), "'/user1/upload/'.concat(" + PARSER.parseExpression("payload.getName()").getExpressionString() + ")");
        sftpOutboundGateway.setRenameExpressionString("'/user1/processed/'.concat(" + PARSER.parseExpression("payload.getName()").getExpressionString() + ")");
        sftpOutboundGateway.setRequiresReply(false);
        sftpOutboundGateway.setOutputChannelName("nullChannel");
        sftpOutboundGateway.setOrder(Ordered.LOWEST_PRECEDENCE);
        sftpOutboundGateway.setAsync(true);
        return sftpOutboundGateway;
    }
Copy after login

thanks for your help!

I looked at the examples in spring integration - sftp to rename or move files in remote server after copying and it helped me a lot

I also checked the official spring intergation sftp documentation

Workaround

I'm not sure how to clear the remote directory on startup. This behavior needs to be debugged on your end. But I can tell you why you are seeing old files. You rename remotely after processing, but local copies of the files are still stored on source.setLocalDirectory(new File("/tmp/local"));. Once you've finished renaming or rebooting, consider cleaning it.

You can also look at SftpStreamingMessageSource to replace your logic: https ://docs.spring.io/spring-integration/reference/sftp/streaming.html

The above is the detailed content of Spring SFTP Intergation: Handling moving files to another directory. For more information, please follow other related articles on the PHP Chinese website!

Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
Popular Tutorials
More>
Latest Downloads
More>
Web Effects
Website Source Code
Website Materials
Front End Template