SAM deployment of lambda (EventSource MQTT) fails with invalid parameter VIRTUAL_HOST

Question

I am struggeling with an issue which appeared out of a sudden between two deployments of our application.

What we are doing is. There is a lambda function which has an EventSource configured. In this case it is a MessageQueue (MQ) Event, listening to a AmazonMQ RabbitMQ Broker. It worked fine for many months but with todays deployment it failed.

Last working deployment: 2022-02-02 11:06:16 UTC+0100

Error:

Resource handler returned message: "Invalid request provided: Invalid parameters: VIRTUAL_HOST (Service: Lambda, Status Code: 400, Request ID:, Extended Request ID: null)" (RequestToken: , HandlerErrorCode: InvalidRequest)

Template excerpt:

    PagespeedCrawlerFunction:
    Type: 'AWS::Serverless::Function'
    Properties:
        CodeUri: .
        Events:
            MQEvent:
                Type: MQ
                Properties:
                    BatchSize: 120
                    Enabled: true
                    Broker: 'arn:aws:mq:us-east-1:11111:broker:cwv-broker:11111'
                    Queues:
                        - 'consumer-queue-name'
                    SourceAccessConfigurations:
                        - Type: BASIC_AUTH
                          URI: 'arn:aws:secretsmanager:us-east-1:1111:secret:global-secrets/rabbitmq-broker-credentials'
                        - Type: VIRTUAL_HOST
                          URI: '/consumervhost'
        FunctionName: 'consumer-v1-prod'
        Handler: handler/consumer.php
        Layers:
            - !Sub 'arn:aws:lambda:${AWS::Region}:209497400698:layer:php-80:16'
            - !Sub 'arn:aws:lambda:${AWS::Region}:403367587399:layer:redis-php-80:11'
        MemorySize: 250
        Policies:
            -
                AWSSecretsManagerGetSecretValuePolicy:
                    SecretArn: 'arn:aws:secretsmanager:us-east-1:11111:secret:global-secrets/rabbitmq-broker-credentials'
            -
                VPCAccessPolicy: {}
            - !Ref CwvMqAccessPolicy
        ReservedConcurrentExecutions: 5
        Runtime: provided.al2
        Timeout: 900
        VpcConfig:
            SecurityGroupIds:
                - !ImportValue MainVPC-DefaultSecurityGroup
            SubnetIds:
                - !ImportValue MainVPC-SubnetPrivateA
                - !ImportValue MainVPC-SubnetPrivateB
    Parameters:
        RetentionDays: 1

Sam version: SAM CLI, version 1.37.0

Deployment script:

sam package \
    --output-template-file /tmp/deploy-stack.yaml \
    --s3-bucket "deployment-resources" \
    --profile "$AWS_PROFILE"

sam deploy \
    --template-file /tmp/deploy-stack.yaml \
    --s3-bucket "deployment-resources" \
    --capabilities CAPABILITY_IAM \
    --stack-name "consumer-prod-v1" \
    --profile "$AWS_PROFILE"

Help is much appreciated.

Answer 1

I managed to solve the issue.

The error is incredible misleading from Cloudformation. Instead of complaining the parameter VIRTUAL_HOST is invalid, it should say: You can not update an already existing EventSource

What I found out after digging in my setup was, that the current BatchSize was set to 50 in our production environment even though it should be 120. To update the BatchSize you need to remove the Event from the template, deploy it, add it again with new BatchSize and deploy again.

Doing exactly that solved the deployment issue.

I was aware of this updating-problem since we use event triggers but this error message totally confused me.

So long story short. If you ever ran into issue with Event-Trigger on Lambda, check first if you are not about to deploy non-updatable changes on the trigger.