ECS Fargate Task in EventBridge fails with ResourceInitializationError

Question

I have created an ECS Fargate Task, which I can manually run. It updates a Dynomodb and I get logs.

Now I want this to run on a schedule. I have setup a scheduled ECS task through EventBridge. However, this does not run.

My looking at the EventBridge logs I can see that the container has been stopped for the following stopped reason:

ResourceInitializationError: unable to pull secrets or registry auth: execution resource 
retrieval failed: unable to retrieve ecr registry auth: service call has been retried 3
time(s): RequestError: send request failed caused by: Post https://api.ecr....

I thought this might be a problem with permissions. However, I tested giving the Task Execution Role full power user permissions and I still get the same error. Could the problem be something else?

Answer 1

I fixed this by enabling auto-assign public IP.

However, to do this, I had to first change from "Capacity provider strategy" - "Use cluster default", to "Launch type" - "FARGATE". Then the option to enable auto-assign public IP became available in the dropdown in the EventBridge UI.

This seems odd to me, because my default capacity provider strategy for my cluster is Fargate. But it is working now.

Answer 2

This is due to a connectivity issue.

The docs say the following:

For tasks on Fargate, in order for the task to pull the container image it must either use a public subnet and be assigned a public IP address or a private subnet that has a route to the internet or a NAT gateway that can route requests to the internet.

So you need to make sure your task has a route to an internet gateway (i.e. it's in a Public subnet) or a NAT gateway.

Alternatively, if your service is in an isolated subnet, you need to create VPC endpoints for ECR and other services you need to call, as described in the docs:

To allow your tasks to pull private images from Amazon ECR, you must create the interface VPC endpoints for Amazon ECR.

When you create a scheduled task, you also specify the networking options. The docs mention this step:

(Optional) Expand Configure network configuration to specify a network configuration. This is required for tasks hosted on Fargate and for tasks using the awsvpc network mode. For Subnets, specify one or more subnet IDs. For Security groups, specify one or more security group IDs. For Auto-assign public IP, specify whether to assign a public IP address from your subnet to the task.

So the networking configuration changed between the manually run task and the scheduled task. Refer to the above to figure out the needed settings for your case.