Apache Camel SFTP get file content null

Question

Java Code

@Component
@RequiredArgsConstructor
public class WireInboundFileListener extends RouteBuilder {

    private final JwtWireTokenService jwtWireTokenService;

    private static final ObjectMapper OBJECT_MAPPER = new ObjectMapper();

    @Value("${wire.ftp.protocol}")
    private String ftpProtocol;

    @Value("${wire.ftp.server}")
    private String ftpServer;

    @Value("${wire.ftp.server.port}")
    private String ftpServerPort;

    @Value("${wire.ftp.username}")
    private String ftpUsername;

    @Value("${wire.ftp.password}")
    private String ftpPassword;

    @Value("${wire.ftp.private.key.file}")
    private String ftpPrivateKeyFile;

    @Value("${wire.ftp.private.key.passphrase}")
    private String privateKeyPassphrase;

    @Value("${wire.file.inbound.dir}")
    private String ftpListenDir;

    @Value("${wire.file.inbound.url}")
    private String inboundUrl;

    @Override
    public void configure() {

        var fromFtpUri = String.format("%s:%s:%s/%s?username=%s&delete=true&antInclude=*.txt",
                ftpProtocol, ftpServer, ftpServerPort, ftpListenDir, ftpUsername);

        log.info("SFTP inbound listen dir : " + ftpListenDir);

        if (Environment.getExecutionEnvironment().equals(Environment.ExecutionEnvironment.AWS)) {
            fromFtpUri += "&privateKeyFile=" + ftpPrivateKeyFile + "&privateKeyPassphrase=" + privateKeyPassphrase;
        } else {
            fromFtpUri += "&password=" + ftpPassword;
        }

        from(fromFtpUri)
                //.delay(10000) event I put delay but still got file content null
                .convertBodyTo(String.class)
                .process(exchange -> {
                    final var requestBody = new HashMap<String, Object>();
                    final var content = exchange.getIn().getBody();
                    final var fileName = exchange.getIn().getHeader("CamelFileName");
                    requestBody.put("content", content);
                    requestBody.put("name", fileName);
                    exchange.getIn().setBody(OBJECT_MAPPER.writeValueAsString(requestBody));
                })
                .to("log:com.test.wire.inbound.listener.SftpRoute")
                .setHeader(Exchange.HTTP_METHOD, constant("POST"))
                .setHeader("Content-Type", constant("application/json"))
                .setHeader("Authorization", method(this, "clientToken"))
                .to(inboundUrl + "?throwExceptionOnFailure=false")
                .log("Response body from wire inbound : ${body}")
                .end();
    }

    public String clientToken() {
        return "Bearer " + jwtWireTokenService.getToken();
    }
}

Success Request

2022-04-20 03:46:47.910  INFO 1 --- [read #6 - Delay] c.test.wire.inbound.listener.SftpRoute  : Exchange[ExchangePattern: InOnly, BodyType: String, Body: {
    "name": "sample-inbound-transfer-20220414024722.txt",
    "content": "file content"
}]

Fail Request

2022-04-21 09:36:54.148  INFO 1 --- [read #4 - Delay] c.test.wire.inbound.listener.SftpRoute  : Exchange[ExchangePattern: InOnly, BodyType: String, Body: {
    "name": "sample-inbound-transfer-20220414024722.txt",
    "content": ""
}]

Main issue

 final var content = exchange.getIn().getBody();// sometimes get null, sometimes can get file contents

When I test to drop a file to SFTP server in local it worked as I expected in Sucess Request because the process to upload seemed fast because it is in local (FileZilla). But when I test to drop a file again to SFTP server that hosts in real server sometimes it works and sometimes it didn't work Fail Request. Seem like the SFTP consuming file issue. Could you please help me on that? Thanks

Answer 1

The file is probably (sometimes) consumed by your Camel route before the file upload is finished.

What you can try is to configure your endpoint to poll the files only if it has exclusive read-lock on the file (i.e. the file is not in-progress or being written); see the readLock parameter.

I think the default is no read lock; you should set readLock=changed...but I do not remember is such mode is also possible on endpoints of SFTP type.