How to define Log Configuration for AWS Batch in Step function definition?

Question

I am trying to deploy a AWS step function where each state machine runs a AWS Batch job. All worked successfully but now I need to store all the logs for these state machines in a specific Cloudwatch log group.

Based on AWS documentation for Batch, I try this snippet in my step function definition in cloudformation template -

*"ContainerOverrides": {

"LogConfiguration": { #also tried logConfiguration
    "LogDriver": "awslogs", #also tried logDriver
    "Options": { #also tried options
       "awslogs-group": "${PipelineLogGroup}",
       "awslogs-stream-prefix": "canonical-"
    }
}

}*

Under the same "ContainerOverrides" tag, "Environment" is defined and is working correctly. For "Log Configuration", I receiving the build error - 'SCHEMA_VALIDATION_FAILED: The field "LogConfiguration" is not supported by Step Functions (same for logConfiguration).

Isn't it possible to define "Log Configuration" of AWS Batch job through Step Function definition?

Answer 1

The "LogConfiguration" is not a part of "ContainerOverrides" in "StateMachine" tag. Rather, it is required to be configured with the Batch job definition.

> PipelineJobDefinition:
>     Type: AWS::Batch::JobDefinition
>     Properties:
>       Type: Container
>       ContainerProperties:
>         Memory: 32768
>         LogConfiguration:
>           LogDriver: 'awslogs'
>           Options: {
>             "awslogs-group": !Ref 'LogGroupCreatedinCF',
>             "awslogs-stream-prefix": "batch"
>           }
>         JobRoleArn: !ImportValue pipeline-DataBatchJobRoleArn
>         Vcpus: 16
>         Image: "...."
>       JobDefinitionName: PipelineJobDefinition
>       RetryStrategy:
>         Attempts: 1