'Azure BlockBlobClient uploadWithResponse file size issue
We are using BlockBlobClient to **upload payload data with ** 20MB file using upload method. Am getting below error. Can some one help me this.
2022-04-05T18:58:49.229-07:00 [APP/PROC/WEB/0] [OUT] Caused by: java.lang.IllegalStateException: io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 16384 byte(s) of direct memory (used: 10471501, max: 10485760) 2022-04-05T18:58:49.229-07:00 [APP/PROC/WEB/0] [OUT] at io.netty.channel.AbstractCoalescingBufferQueue.releaseAndCompleteAll(AbstractCoalescingBufferQueue.java:351) ~[netty-transport-4.1.48.Final.jar:4.1.48.Final] 2022-04-05T18:58:49.229-07:00 [APP/PROC/WEB/0] [OUT] at io.netty.channel.AbstractCoalescingBufferQueue.releaseAndFailAll(AbstractCoalescingBufferQueue.java:207) ~[netty-transport-4.1.48.Final.jar:4.1.48.Final] 2022-04-05T18:58:49.229-07:00 [APP/PROC/WEB/0] [OUT] at io.netty.handler.ssl.SslHandler.releaseAndFailAll(SslHandler.java:1830) ~[netty-handler-4.1.48.Final.jar:4.1.48.Final] 2022-04-05T18:58:49.229-07:00 [APP/PROC/WEB/0] [OUT] at io.netty.handler.ssl.SslHandler.setHandshakeFailure(SslHandler.java:1809) ~[netty-handler-4.1.48.Final.jar:4.1.48.Final] 2022-04-05T18:58:49.229-07:00 [APP/PROC/WEB/0] [OUT] at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:750) ~[netty-transport-4.1.48.Final.jar:4.1.48.Final] 2022-04-05T18:58:49.229-07:00 [APP/PROC/WEB/0] [OUT] at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:742) ~[netty-transport-4.1.48.Final.jar:4.1.48.Final] 2022-04-05T18:58:49.229-07:00 [APP/PROC/WEB/0] [OUT] at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.flush(CombinedChannelDuplexHandler.java:531) ~[netty-transport-4.1.48.Final.jar:4.1.48.Final] 2022-04-05T18:58:49.229-07:00 [APP/PROC/WEB/0] [OUT] at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:125) ~[netty-transport-4.1.48.Final.jar:4.1.48.Final] 2022-04-05T18:58:49.229-07:00 [APP/PROC/WEB/0] [OUT] at io.netty.channel.CombinedChannelDuplexHandler.flush(CombinedChannelDuplexHandler.java:356) ~[netty-transport-4.1.48.Final.jar:4.1.48.Final] 2022-04-05T18:58:49.229-07:00 [APP/PROC/WEB/0] [OUT] at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:750) ~[netty-transport-4.1.48.Final.jar:4.1.48.Final] 2022-04-05T18:58:49.229-07:00 [APP/PROC/WEB/0] [OUT] at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:742) ~[netty-transport-4.1.48.Final.jar:4.1.48.Final] 2022-04-05T18:58:49.229-07:00 [APP/PROC/WEB/0] [OUT] at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:728) ~[netty-transport-4.1.48.Final.jar:4.1.48.Final] 2022-04-05T18:58:49.229-07:00 [APP/PROC/WEB/0] [OUT] at reactor.netty.channel.MonoSendMany$SendManyInner.run(MonoSendMany.java:325) ~[reactor-netty-core-1.0.4.jar:1.0.4] 2022-04-05T18:58:49.229-07:00 [APP/PROC/WEB/0] [OUT] at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164) ~[netty-common-4.1.48.Final.jar:4.1.48.Final] 2022-04-05T18:58:49.229-07:00 [APP/PROC/WEB/0] [OUT] at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472) ~[netty-common-4.1.48.Final.jar:4.1.48.Final] 2022-04-05T18:58:49.229-07:00 [APP/PROC/WEB/0] [OUT] at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384) ~[netty-transport-native-epoll-4.1.48.Final-linux-x86_64.jar:4.1.48.Final] 2022-04-05T18:58:49.229-07:00 [APP/PROC/WEB/0] [OUT] at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[netty-common-4.1.48.Final.jar:4.1.48.Final] 2022-04-05T18:58:49.229-07:00 [APP/PROC/WEB/0] [OUT] at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[netty-common-4.1.48.Final.jar:4.1.48.Final] 2022-04-05T18:58:49.229-07:00 [APP/PROC/WEB/0] [OUT] at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[netty-common-4.1.48.Final.jar:4.1.48.Final] 2022-04-05T18:58:49.229-07:00 [APP/PROC/WEB/0] [OUT] ... 1 common frames omitted 2022-04-05T18:58:49.229-07:00 [APP/PROC/WEB/0] [OUT] at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:99) ~[reactor-core-3.4.3.jar:3.4.3] 2022-04-05T18:58:49.229-07:00 [APP/PROC/WEB/0] [OUT] at reactor.core.publisher.Mono.block(Mono.java:1703) ~[reactor-core-3.4.3.jar:3.4.3] 2022-04-05T18:58:49.229-07:00 [APP/PROC/WEB/0] [OUT] at com.azure.storage.common.implementation.StorageImplUtils.blockWithOptionalTimeout(StorageImplUtils.java:128) ~[azure-storage-common-12.12.0.jar:na] 2022-04-05T18:58:49.229-07:00 [APP/PROC/WEB/0] [OUT] at com.azure.storage.blob.specialized.BlockBlobClient.uploadWithResponse(BlockBlobClient.java:340) ~[azure-storage-blob-12.12.0.jar:na] 2022-04-05T18:58:49.229-07:00 [APP/PROC/WEB/0] [OUT] at com.azure.storage.blob.specialized.BlockBlobClient.upload(BlockBlobClient.java:262) ~[azure-storage-blob-12.12.0.jar:na] 2022-04-05T18:58:49.229-07:00 [APP/PROC/WEB/0] [OUT] at com.azure.storage.blob.specialized.BlockBlobClient.upload(BlockBlobClient.java:233) ~[azure-storage-blob-12.12.0.jar:na]
Implemented Code to upload
public String uploadToBlob(String payload, String fileName) throws AzureStorageException {
BlockBlobClient blockBlobClient = blobContainerClient.getBlobClient(fileName).getBlockBlobClient();
try (ByteArrayInputStream dataStream = new ByteArrayInputStream(payload.getBytes(UTF_8))) {
blockBlobClient.upload(dataStream, payload.length());
return blockBlobClient.getBlobUrl();
} catch (Exception e) {
log.error(format("Error occurred while uploadToBlob %s: %s %s:", fileName, WITH_EXCEPTION, e.getMessage()));
throw new AzureStorageException(e.getMessage(), e);
}
}
Any suggestion on this will help us to resolve the issue.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
