'Recursive list s3 bucket contents with AWS CLI

How can I recursively list all all the contents of a bucket using the AWS CLI similar to using find . on Unix.

aws s3 ls s3://MyBucket --recursive complains with unknown option.

http://docs.aws.amazon.com/cli/latest/reference/s3/index.html#directory-and-s3-prefix-operations claims that --recursive is a valid parameter.

aws-cli

Solution 1:^[1]

aws s3 ls s3://MyBucket --recursive works fine for me.

Try updating your AWS CLI. My version is aws-cli/1.6.2

aws --version

Solution 2:^[2]

With recent AWS CLI versions, --recursive option is supported.

You can list recursively all the files under a bucket named MyBucket using following command:

aws s3 ls s3://MyBucket/ --recursive

You can list recursively all the files under a folder named MyFolder in the bucket, using following command:

aws s3 ls s3://MyBucket/MyFolder/ --recursive

As @Francisco Cardoso said, the final / is very important. It allows to list the content of the folder instead of the folder itself

For more information, see: https://docs.aws.amazon.com/cli/latest/reference/s3/ls.html

Solution 3:^[3]

I am not able to interpret the link you referred properly: http://docs.aws.amazon.com/cli/latest/reference/s3/index.html#directory-and-s3-prefix-operations

However, I was able to make --recursive option work with respect to this link: http://docs.aws.amazon.com/cli/latest/reference/s3/index.html#single-local-file-and-s3-object-operations

as per this link, cp, mv and rm supports --recursive option.

The one that you are trying is ls.

I tried using cp and rm with --recursive option and it is working fine.

Solution 4:^[4]

You can not list recursively all the contents of a bucket via -

aws s3 ls s3://MyBucket

To list object from a folder you need to execute command as -

aws s3 ls s3://MyBucket/MyFolder/

This above command lists object that reside inside folder named MyFolder.

To get an objects list from such a logical hierarchy from Amazon S3, you need specify the full key name for the object in the GET operation.

--recursive Command is performed on allfiles or objects under the specified directory or prefix.

Thanks

Solution 5:^[5]

Below one line bash script is able to perform:- how to list all S3 buckets with their objects recursively, list bucket name and count objects also.

/usr/bin/sudo /usr/local/bin/aws s3 ls |awk '{print $NF}'| while read l;do echo -e "#######---$l objects---##########\n\n";/usr/bin/sudo /usr/local/bin/aws s3 ls $l|nl;done

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution	Source
Solution 1	wisbucky
Solution 2
Solution 3	slayedbylucifer
Solution 4
Solution 5

'Recursive list s3 bucket contents with AWS CLI

Solution 1:[1]

Solution 2:[2]

Solution 3:[3]

Solution 4:[4]

Solution 5:[5]