Signature Error while updating S3 object metadata through boto3

Question

I have a lambda function that takes S3 object from S3 events and updates it with the custom metadata.

Here is the boto3 script:

import json
import boto3

s3 = boto3.resource('s3')

def lambda_handler(event, context):
    key = event['Records'][0]['s3']['object']['key']
    key_name = key.split('/')
    bucket = event['Records'][0]['s3']['bucket']['name']
    print(key)
    print(bucket)
    s3_object = s3.Object(bucket, key)
    s3_object.metadata.update({'Cache-Control':'no-cache'})
    s3_object.copy_from(CopySource={'Bucket':bucket, 'Key':key}, Metadata=s3_object.metadata, MetadataDirective='REPLACE')

When I run the script, it gives me the following error:

An error occurred (SignatureDoesNotMatch) when calling the CopyObject operation: The request signature we calculated does not match the signature you provided. Check your key and signing method.: ClientError

(Note: I have given sufficient permission on lambda function role)

Do I need to create Signature here?

Answer 1

John Rotenstein's answers works, but only ONCE. Running the same script twice reproduces the error OP had.

I believe this is related to why:

User-defined metadata is a set of key-value pairs. Amazon S3 stores user-defined metadata keys in lowercase.

https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingMetadata.html

Notice that although Amazon stores keys as lowercase, OP is using upper case. Changing the "Cache-Control" to the lowercase "cache-control" will allow you to update metadata on objects where "cache-control" is already a metadatum entry.

I have tested the case sensitive issue and am able to get things working without the capitalization of the key, and reproduce the issue when the key capitalization is reintroduced.

There appears to be a discrepancy on how the signature is calculated on the client end and on the API backend at AWS. The capitalization issue only expresses itself when the metadatum entry is pre-existing on the object. This might warrant raising a ticket with AWS regarding the boto3 library.