S3 Ninja + AWS Java SDK Does Not Connect

Question

The Issue

I want to use S3 Ninja to emulate S3 in my own network for testing and demo purposes. I am using Java SDK (with Scala).

My storage code works fine if it is connected to the real s3 but the same code does not work.

  override def store(bucketName: String, keyName: String, source: File): 
  BlobStoreResult = {
        val iAm = "store"
        val transferManager = TransferManagerBuilder.standard
        .withS3Client(createS3Client())
        .build
        val fileName = source.toPath.getFileName.toString
        try {
        val transfer = transferManager.upload(bucketName, keyName, source)

This results in error message logged like this:

c.a.request - Sending Request: PUT http://test-bucket.localhost:9444 /png-test.png 
Headers: (amz-sdk-invocation-id: bb866d5c-2ac7-4b30-579b-01df10b96e81, Content-Length: 
924, Content-MD5: Trqia4kjznKAi0p/v3JesA==, Content-Type: image/png, User-Agent: aws-
sdk-java/1.12.53 Windows_10/10.0 Eclipse_OpenJ9_VM/openj9-0.27.0 java/11.0.12 
scala/2.13.6 groovy/2.5.14 vendor/International_Business_Machines_Corporation 
cfg/retry-mode/legacy com.amazonaws.services.s3.transfer.TransferManager/1.12.53, ) 

c.a.a.AWS4Signer - AWS4 Canonical Request: '"PUT
/png-test.png

Answer 1

The Root Cause

The S3 Java SDK defaults to virtual hosted url strategy if you altered the end-point. S3 Ninja requires the path style access strategy.

The Fix

This can be easily modified by an addition of .withPathStyleAccessEnabled(true) in your client builder's build call chain.

  val client = AmazonS3ClientBuilder.standard()
    .withCredentials(credentialProvider)
    .withEndpointConfiguration(ep)
    .withPathStyleAccessEnabled(true)