How to put data from an OutputStream into a ByteBuffer?

Question

In Java I need to put content from an OutputStream (I fill data to that stream myself) into a ByteBuffer. How to do it in a simple way?

Answer 1

There is a more efficient variant of @DJClayworth's answer.

As @seh correctly noticed, ByteArrayOutputStream.toByteArray() returns a copy of the backing byte[] object, which may be inefficient. However, the backing byte[] object as well as the count of the bytes are both protected members of the ByteArrayOutputStream class. Hence, you can create your own variant of the ByteArrayOutputStream exposing them directly:

public class MyByteArrayOutputStream extends ByteArrayOutputStream {
  public MyByteArrayOutputStream() {
  }

  public MyByteArrayOutputStream(int size) {
    super(size);
  }

  public int getCount() {
    return count;
  }

  public byte[] getBuf() {
    return buf;
  }
}

Using this class is easy:

MyByteArrayOutputStream out = new MyByteArrayOutputStream();
fillTheOutputStream(out);
return new ByteArrayInputStream(out.getBuf(), 0, out.getCount());

As a result, once all the output is written the same buffer is used as the basis of an input stream.

Answer 2

Though the above-mention answers solve your problem, none of them are efficient as you expect from NIO. ByteArrayOutputStream or MyByteArrayOutputStream first write the data into a Java heap memory and then copy it to ByteBuffer which greatly affects the performance.

An efficient implementation would be writing ByteBufferOutputStream class yourself. Actually It's quite easy to do. You have to just provide a write() method. See this link for ByteBufferInputStream.

Answer 3

Try using PipedOutputStream instead of OutputStream. You can then connect a PipedInputStream to read the data back out of the PipedOutputStream.

Answer 4

You say you're writing to this stream yourself? If so, maybe you could implement your own ByteBufferOutputStream and plug n' play.

The base class would look like so:

public class ByteBufferOutputStream extends OutputStream {
    //protected WritableByteChannel wbc; //if you need to write directly to a channel
    protected static int bs = 2 * 1024 * 1024; //2MB buffer size, change as needed
    protected ByteBuffer bb = ByteBuffer.allocate(bs);

    public ByteBufferOutputStream(...) {
        //wbc = ... //again for writing to a channel
    }

    @Override
    public void write(int i) throws IOException {
        if (!bb.hasRemaining()) flush();
        byte b = (byte) i;
        bb.put(b);
    }

    @Override
    public void write(byte[] b, int off, int len) throws IOException {
        if (bb.remaining() < len) flush();
        bb.put(b, off, len);
    }

    /* do something with the buffer when it's full (perhaps write to channel?)
    @Override
    public void flush() throws IOException {
        bb.flip();
        wbc.write(bb);
        bb.clear();
    }

    @Override
    public void close() throws IOException {
        flush();
        wbc.close();
    }
    /*
}

Answer 5

// access the protected member buf & count from the extend class

class ByteArrayOutputStream2ByteBuffer extends ByteArrayOutputStream {
    public ByteBuffer toByteBuffer() {
        return ByteBuffer.wrap(buf, 0, count);
    }
}

Answer 6

See efficient implementation of ByteBuffer backed OutputStream with dynamic re-allocation.

/**
 * Wraps a {@link ByteBuffer} so it can be used like an {@link OutputStream}. This is similar to a
 * {@link java.io.ByteArrayOutputStream}, just that this uses a {@code ByteBuffer} instead of a
 * {@code byte[]} as internal storage.
 */
public class ByteBufferOutputStream extends OutputStream {

    private ByteBuffer wrappedBuffer;
    private final boolean autoEnlarge;

https://gist.github.com/hoijui/7fe8a6d31b20ae7af945