'how to calculate size of S3 bucket if size is more than GB using java
I am not able to calculate S3 bucket size if it is more than 300GB. I have used below code to calculate no of objects and bucket size. it is working for bucket less than 300gb size. I need to upload till 4TB and want to verify the no. of objects and size.
long totalSize = 0;
int totalItems = 0;
List<S3Object> s3Obj = new ArrayList<S3Object>();
ListObjectsResult result = null;
do {
if (result == null)
result = s3JerseyClient.listObjects(bucketName);
else
result = s3JerseyClient.listMoreObjects(result);
s3Obj.addAll(result.getObjects());
for (S3Object objectSummary : result.getObjects()) {
totalSize += objectSummary.getSize();
totalItems++;
}
} while (result.isTruncated());
if no of objects more than 600k above code is sometimes giving OOM error or it is running more than an hour
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
Solution | Source |
---|