Terraform Data Source: aws_s3_object can't get object from S3 bucket in another account

Question

Hi Stack overflow community,

I have some Terraform code that needs access to an object in a bucket that is located in a different AWS account than the one I'm deploying the Terraform to.

The AWS S3 bucket is in us-west-2 and I'm deploying the Terraform in us-east-1 (I don't think this should matter).

I set up the following bucket level policy in the S3 bucket:

{
    "Version": "2012-10-17",
    "Id": "Policy1",
    "Statement": [
        {
            "Sid": "Stmt1",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::<aws-account-number-where-terraform-will-be-deployed>:user/<user-deploying-terraform>"
            },
            "Action": [
                "s3:GetObject*",
                "s3:List*"
            ],
            "Resource": [
                "arn:aws:s3:::<bucket-name>/*",
                "arn:aws:s3:::<bucket-name>"
            ]
        },
    ]
}

When I run the following AWS CLI command I'm able to get the bucket object using the user that will be deploying the Terraform:

aws s3api get-object --bucket "<bucket-name>" --key "<path-to-file>" "test.txt"

But when I run the following Terraform code:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "= 4.6.0"
    }
  }
}

data "aws_s3_object" "this" {
  bucket = "<bucket-name>"
  key    = "<path-to-file>"
}

output "test" {
    value = data.aws_s3_object.this.body
}

I get the following error:

Error: failed getting S3 Bucket (<bucket-name>) Object (<path-to-file>): BadRequest: Bad Request
    status code: 400, request id: <id>, host id: <host-id>

  with data.aws_s3_object.challenge_file,
  on main.tf line 10, in data "aws_s3_object" "this":
  10: data "aws_s3_object" "this" {

Answer 1

The provider configuration, as specified by AWS and Hashicorp, uses a single set of credentials, region, etc. You need a second provider configuration with an alias for the other region.

provider "aws" {
  alias  = "us-west-2"
  region = "us-west-2"
}

data "aws_s3_object" "this" {
  provider = aws.us-west-2
  bucket   = "<bucket-name>"
  key      = "<path-to-file>"
}

if your supplied credentials are not sufficient for permissions to retrieve information about the bucket in the other account, then the provider configuration block will also need separate credentials.