AccessDenied for ListObjectsV2 operation for S3 bucket

Question

During GitlabCi I got: "fatal error: An error occurred (AccessDenied) when calling the ListObjectsV2 operation: Access Denied"

My bucket policy :

{
"Version": "2008-10-17",
"Statement": [
    {
        "Sid": "AllowPublicRead",
        "Effect": "Allow",
        "Principal": {
            "AWS": "*"
        },
        "Action": "s3:*",
        "Resource": "arn:aws:s3:::BUCKET-NAME/*"
    }
]

}

In gitlabCI settings set:

• AWS_ACCESS_KEY_ID: YOUR-AWS-ACCESS-KEY-ID
• AWS_SECRET_ACCESS_KEY: YOUR-AWS-SECRET-ACCESS-KEY
• S3_BUCKET_NAME: YOUR-S3-BUCKET-NAME
• DISTRIBUTION_ID: CLOUDFRONT-DISTRIBUTION-ID

My .gitlab-ci.yml

image: docker:latest

stages:
  - build
  - deploy

build:
  stage: build
  image: node:8.11.3
script:
  - export API_URL="d144iew37xsh40.cloudfront.net"
  - npm install
  - npm run build
  - echo "BUILD SUCCESSFULLY"
artifacts:
   paths:
    - public/
expire_in: 20 mins
environment:
name: production
only:
   - master
deploy:
  stage: deploy
  image: python:3.5
dependencies:
   - build
script:
  - export AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID
  - export AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY
  - export S3_BUCKET_NAME=$S3_BUCKET_NAME
  - export DISTRIBUTION_ID=$DISTRIBUTION_ID
  - pip install awscli --upgrade --user
  - export PATH=~/.local/bin:$PATH
  - aws s3 sync --acl public-read --delete public $S3_BUCKET_NAME
  - aws cloudfront create-invalidation --distribution-id 
$DISTRIBUTION_ID --paths '/*'
  - echo "DEPLOYED SUCCESSFULLY"

environment:
   name: production
only:
  - master

Answer 1

I'm not sure the accepted answer is actually acceptable, as it simply allows all operations on the bucket. Also the Sid is misleading... ;-)

This AWS article mentions the required permissions for aws s3 sync.

This is how a corresponding policy looks like:

{
"Version": "version_id",
"Statement": [
    {
        "Sid": "AllowBucketSync",
        "Effect": "Allow",
        "Action": [
            "s3:GetObject", 
            "s3:PutObject", 
            "s3:ListBucket"
        ],
        "Resource": [
            "arn:aws:s3:::BUCKET-NAME",
            "arn:aws:s3:::BUCKET-NAME/*"
        ]
    }
] }

Answer 2

I had this problem recently. No matter what I did, no matter what permissions I provided, I kept getting "An error occurred (AccessDenied) when calling the ListObjectsV2 operation: Access Denied" when running aws s3 ls <bucket>

I had forgotten that I have multiple aws profiles configured in my environment. The aws command was using the default profile, which has a different set of access keys. I had to specify the --profile flag to the command:

aws s3 ls <bucket> --profile <correct profile>

That worked. It's a niche situation, but maybe it'll help someone out.

Answer 3

I got "AccessDenied" errors, too, even though the policy was correct. I gave mrbranden's solution a try though I only have one (the default) credentials configured. And lo and behold,

$ aws s3 ls <bucket> --profile=default

made it work!

My aws --version is aws-cli/1.18.69 Python/3.8.5 Linux/5.4.0-1035-aws botocore/1.16.19

Answer 4

Principle is required now and it should look like this:

{
"Version": "2012-10-17",
"Statement": [
    {
        "Sid": "AllowBucketSync",
        "Effect": "Allow",
        "Principal": "*",
        "Action": [
            "s3:GetObject",
            "s3:PutObject",
            "s3:ListBucket"
        ],
        "Resource": [
            "arn:aws:s3:::YOURBUCKET",
            "arn:aws:s3:::YOURBUCKET/*"
        ]
    }
]

}

Answer 5

I got this as a rather misleading error message when I mistakenly used the full domain name with the s3:// prefix to select the bucket to operate on, like s3://s3.amazonaws.com/bucket_name. Switching to s3://bucket_name fixed the problem.

Answer 6

For Amazon users who have enabled MFA, please use this: aws s3 ls s3://bucket-name --profile mfa.

And prepare the profile mfa first by running aws sts get-session-token --serial-number arn:aws:iam::123456789012:mfa/user-name --token-code 797395 --duration 129600. (replace 123456789012, user-name and 797395).