JSch's ChannelSftp doesn't download more than one file at a time

Question

It's too frustrating that JSch doesn't throw exceptions with decent messages. I'm trying to download a set of files as InputStreams. The code to download the file is pretty simple:

  @Override
  @SneakyThrows
  public InputStream getInputStream(String path) {
    return channelSftp.get(path);
  }

I have this list of file URLs that I'm using to download InputStream and convert it to DataPages:

    List<DataPage> dataPages =
        files.stream()
            .map(
                fileName -> {
                  String fileURL = folderUrl[0] + "/" + fileName;
                  return client.getInputStream(fileURL);
                })
            .map(dataPageFunction)
            .collect(Collectors.toList());

The first file is downloaded successfully. The problem occurs when we get the second file. I enabled log to discover if I find out anything, but the only thing I got is:

2022-04-13T10:58:27.916Z INFO [Connect thread localhost session] i.s.c.e.c.SftpClient$JschLogger:158 Caught an exception, leaving main loop due to Socket closed

I tried a different way to get files:

  @SneakyThrows
  public Stream<InputStream> getInputStreams(String folderURL, String filePattern) {
    Vector<ChannelSftp.LsEntry> lsEntries = channelSftp.ls(folderURL + "/" + filePattern);
    return lsEntries.stream()
        .map(
            entry -> {
              Optional<InputStream> is = Optional.empty();
              try {
                is = Optional.of(channelSftp.get(folderURL + "/" + entry.getFilename()));
              } catch (SftpException e) {
                log.error(e.getMessage());
              }
              return is;
            })
        .filter(Optional::isPresent)
        .map(Optional::get);
  }

And as noticed previously, the first file was downloaded successfully. There were three files in total, the channelSftp.get method failed for second and third file.

The printed logs were:

2022-04-13T12:08:06.864Z ERROR [Test worker] i.s.c.e.c.SftpClient:132 
2022-04-13T12:08:08.316Z ERROR [Test worker] i.s.c.e.c.SftpClient:132

and the stack trace was nothing but:

4: 
    at app//com.jcraft.jsch.ChannelSftp._stat(ChannelSftp.java:2227)
    at app//com.jcraft.jsch.ChannelSftp._stat(ChannelSftp.java:2242)
    at app//com.jcraft.jsch.ChannelSftp.ls(ChannelSftp.java:1592)
    at app//com.jcraft.jsch.ChannelSftp.ls(ChannelSftp.java:1553)
.
.
.

I'd like to include my open and close config just incase:

  @SneakyThrows
  public void open() {
    JSch jsch = new JSch();
    JSch.setLogger(new JschLogger());
    Properties config = new Properties();
    config.put("StrictHostKeyChecking", "no");
    if (StringUtils.isNoneBlank(privateKey)) {
      privateKey = SSHUtils.toRSA(privateKey, passphrase);
      jsch.addIdentity(
          "",
          privateKey.getBytes(),
          null,
          Objects.nonNull(passphrase) ? passphrase.getBytes() : null);
    }

    session = jsch.getSession(username, server, Objects.nonNull(port) ? port : DEFAULT_PORT);
    session.setConfig(config);
    if (StringUtils.isNoneBlank(password)) session.setPassword(password);
    session.connect();
    channelSftp = (ChannelSftp) session.openChannel("sftp");
    channelSftp.connect();
  }

  @SneakyThrows
  public void close() {
    if (Objects.nonNull(channelSftp) && channelSftp.isConnected()) channelSftp.disconnect();
    if (Objects.nonNull(session) && session.isConnected()) session.disconnect();
  }

Answer 1

I learned that JSch doesn't go that well with streams. This problem can be fixed by simply looping over lsEntries.

  @SneakyThrows
  public List<InputStream> getInputStreams(String folderURL, String filePattern) {
    Vector<ChannelSftp.LsEntry> lsEntries = channelSftp.ls(folderURL + "/" + filePattern);
    List<InputStream> inputStreams = new ArrayList<>();
    for (LsEntry entry : lsEntries) {
      try {
        inputStreams.add(channelSftp.get(folderURL + "/" + entry.getFilename()));
      } catch (SftpException e) {
        log.error(e.getMessage());
      }
    }
    return inputStreams;
  }