how to use aws secret manager values of a postgres database in terraform ecs task definition

Question

I have the settings.py file below of a django application using terraform , docker compose and im trying to get the value of the database stored in aws secret manager in ecs task definition

settings.py

DATABASES = {
    "default": {
        "ENGINE": "django.db.backends.postgresql_psycopg2",
        "NAME": os.environ.get("POSTGRES_DB"),
        "USER": os.environ.get("POSTGRES_USER"),
        "PASSWORD": os.environ.get("POSTGRES_PASSWORD"),
        "HOST": os.environ.get("POSTGRES_HOST"),
        "PORT": 5432,
    }
}

task definition

resource "aws_ecs_task_definition" "ecs_task_definition" {
  family                   = "ecs_container"
  network_mode             = "awsvpc"
  requires_compatibilities = ["FARGATE"]
  cpu                      = 256                      
  memory                   = 512
  execution_role_arn       = data.aws_iam_role.fargate.arn
  task_role_arn            = data.aws_iam_role.fargate.arn
  container_definitions = jsonencode([
    {
      "name"         : "ecs_test_backend",
      "image"        : "${aws_ecr_repository.ecr_repository.repository_url}:latest",
      "cpu"          : 256,
      "memory"       : 512,
      "essential"    : true,
      "portMappings" : [
        {
          containerPort = 8000
        }
      ],
      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-region": "us-east-1",
          "awslogs-group": "/ecs/ecs_test_backend",
          "awslogs-stream-prefix": "ecs"
        }
      }
      "environment"   : [
        {
          "name": "POSTGRES_DB",
          "value": "${var.POSTGRES_NAME}" <=== HERE
        },
        {
          "name": "POSTGRES_PASSWORD",
          "value": "${var.POSTGRES_PASSWORD}"  <=== HERE
        },
        {
          "name": "POSTGRES_USERNAME",
          "value": "${var.POSTGRES_USERNAME}"  <=== HERE
        },
        {
          "name": "POSTGRES_PORT",
          "value": "${var.POSTGRES_PORT}"  <=== HERE
        },
        {
          "name": "POSTGRES_HOST",
          "value": "${var.POSTGRES_HOST}"  <=== HERE
        },
      ]
    }
  ])
}

variables

variable "POSTGRES_PASSWORD" {
    type = string
    default = "somepassword"
}

The variables above is the same used while creating the postgres rds instance.

The configuration below var.XXX does not seem to work as the task logs return psycopg2.OperationalError: FATAL: password authentication failed for user "root"

It probably because its not able to read the value.

Is the above the correct way to grab value from AWS Secret Manager using Terraform and ECS?

Answer 1

You should use the sensitive data handling feature of ECS tasks, documented here.

You would move the environment variables from the environment block of your task definition to the secrets block, and give the ARN of the secret instead of the value. The ECS container will then read those secrets when it starts your container, and set them in the container's environment.

Answer 2

I have been looking and I couldn't find any good examples to achieve this in the proper way, and while the response from @Mark B adds some context is true that some code snippets can be helpful since sometimes is more complicated to code it (with all the requirements) than the theoretical explanation. With that said I will split into pieces what you need to complete:

The parameter store

You need to save your secrets somewhere and the parameters store is a good place for it according AWS best practices:

resource "aws_ssm_parameter" "main" {
  for_each    = var.parameters
  name        = "/path/${var.app_name}/${each.key}"
  description = "Secrets for application ${var.app_name}"
  type        = "SecureString"
  value       = each.value
}

variable "parameters" {
  type = map(any)
}

variable "app_name" {
  type = string
}

output "arns" {
  value = [for k, v in var.parameters : { name = k, valueFrom = aws_ssm_parameter.main[k].arn }]
}

(The code above creates several parameters)

Be sure of adding the correct permissions to your task execution role:

resource "aws_iam_role_policy" "parameter_policy" {
  name = "mypolicy"
  role = aws_iam_role.your_role.id

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = [
          "ssm:GetParameters",
        ]
        Effect   = "Allow"
        Resource = ["*"]
      },
    ]
  })
}

Be aware that the above is just an example, to follow best practices be sure to limit the resources to the arn(s) of the parameters created above.

Finally how to use it

container_definitions = <<TASK_DEFINITION
[
  {
    "name": "hello-world",
    "image": "nginxdemos/hello",
    "cpu": 1024,
    "memory": 2048,
    "essential": true,
    "secrets": ${jsonencode(module.hello-world-secrets.arns)},
    "logConfiguration": {
      "logDriver": "awslogs",
      "options": {
        "awslogs-region": "your_region",
        "awslogs-group": "hello_world",
        "awslogs-stream-prefix": "hello_world"
      }
    }
  }
]
TASK_DEFINITION

module "hello-world-secrets" {
  source   = "./modules/secrets"
  app_name = "hello-world"
  parameters = {
    hello = "world"
  }
}

You need to decide in which point create the secret / store the secret, e.g. on your CI/CD inject your secret and create the parameter store resource.

Hopefully, this help someone else looking for a way of achieving this.