How can I access S3 resources using AWS SDK.NET by currently user's Security key, access key and session token found by IAM role from metadata

Question

I want to access S3 bucket. But my client doesn't want to pass Security key and access key through Enviroment variables. He is running our code on EC2 instance in AWS. Hence, he wants the code to written so that it automatically finds the credentials using his IAM role.

I am able to get the AWS Security key, Access Key and Security Token using Metadata information but unfortunately when I use these credentials, I get an error message, Key not found in AWS. I tried to hit the http://169.254.169.254/latest/meta-data/iam/security-credentials/MyRole to get AWS security key, access key and session token. With these 3 I am writing below code which is blowing up:

            var crede = new SessionAWSCredentials(_accessKey, _secretKey, _token);

            var s3Client = new AmazonS3Client(_accessKey, _secretKey, _token, config);

            var response = await s3Client.GetObjectAsync(request);

            return response.ResponseStream;


The specified key does not exist., stack:   at Amazon.Runtime.Internal.HttpErrorResponseExceptionHandler.HandleExceptionStream(IRequestContext requestContext, IWebResponseData httpErrorResponse, HttpErrorResponseException exception, Stream responseStream)
   at Amazon.Runtime.Internal.HttpErrorResponseExceptionHandler.HandleExceptionAsync(IExecutionContext executionContext, HttpErrorResponseException exception)
   at Amazon.Runtime.Internal.ExceptionHandler`1.HandleAsync(IExecutionContext executionContext, Exception exception)
   at Amazon.Runtime.Internal.ErrorHandler.ProcessExceptionAsync(IExecutionContext executionContext, Exception exception)
   at Amazon.Runtime.Internal.ErrorHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.EndpointDiscoveryHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.EndpointDiscoveryHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.CredentialsRetriever.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.RetryHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.RetryHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.S3.Internal.AmazonS3ExceptionHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.ErrorCallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.MetricsHandler.InvokeAsync[T](IExecutionContext executionContext)
   at awstest.Program.Main(String[] args) in C:\Users\VikasKumar\source\repos\ConsoleApp1\awstest\Program.cs:line 99

Answer 1

Finally able to resolve this :

    private static ImmutableCredentials FetchCredentials()
    {
        var securityCredentials = EC2InstanceMetadata.IAMSecurityCredentials;
        if (securityCredentials == null)
            throw new AmazonServiceException("Unable to get IAM security credentials from EC2 Instance Metadata Service.");

        string firstRole = null;
        foreach (var role in securityCredentials.Keys)
        {
            firstRole = role;
            break;
        }

        if (string.IsNullOrEmpty(firstRole))
            throw new AmazonServiceException("Unable to get EC2 instance role from EC2 Instance Metadata Service.");

        var metadata = securityCredentials[firstRole];
        if (metadata == null)
            throw new AmazonServiceException("Unable to get credentials for role \"" + firstRole + "\" from EC2 Instance Metadata Service.");

        return new ImmutableCredentials(metadata.AccessKeyId, metadata.SecretAccessKey, metadata.Token);
    }

And calling of that FetchCredentials:

            var credentials = FetchCredentials();
        
            var config = new AmazonS3Config
            {
                RegionEndpoint = RegionEndpoint.GetBySystemName("ap-southeast-2")
            };


            var s3Client = new AmazonS3Client(credentials.AccessKey, credentials.SecretKey, credentials.Token, config);
            Console.Write("temp credentials created s3 client.");
            var request = new GetObjectRequest
            {
                BucketName = "opensparkzstorage",
                Key = "qa/outbound/mys3filename.csv"
            };

            var listReq = new ListObjectsRequest
            {
                BucketName = "mys3bucketname"
            };

            var response = await s3Client.ListObjectsAsync(listReq);

            foreach (S3Object entry in response.S3Objects)
            {
                Console.Write(entry.Key);
            }