How may I scale the dynamodb write capacity on a schedule?

Question

I have scheduled tasks that will be writing large payloads to a specific dynamodb table

My dynamodb is created through terraform, my issue is when I try to autoscale it, it takes some time to provision.

I tried adding an AWS APP auto scale to run at midnight, right when my heavy write requests come in but it's still throttling most of the requests as I see on the graph in the management console

How can I properly scale my write requests on a schedule?

resource "aws_dynamodb_table" "test-dynamodb-table" {
  name           = "test-dynamodb-table"
  billing_mode   = "PROVISIONED
  read_capacity  = "1"
  write_capacity = "1"
  hash_key       = "name"
  range_key      = "address"

  attribute {
    name = "name"
    type = "S"
  }

  attribute {
    name = "address"
    type = "S"
  }
}

resource "aws_appautoscaling_target" "dynamodb" {
  max_capacity       = 200
  min_capacity       = 1
  resource_id        = "table/test-dynamodb-table"
  scalable_dimension = "dynamodb:table:WriteCapacityUnits"
  service_namespace  = "dynamodb"
}

resource "aws_appautoscaling_scheduled_action" "dynamodb" {
  name               = "dynamodb"
  service_namespace  = aws_appautoscaling_target.dynamodb.service_namespace
  resource_id        = aws_appautoscaling_target.dynamodb.resource_id
  scalable_dimension = aws_appautoscaling_target.dynamodb.scalable_dimension
  schedule           = "cron(0 12 * * ? *)"

  scalable_target_action {
    min_capacity = 100
    max_capacity = 200
  }
}

Answer 1

You can change your auto scaling policy to have the new desired value as the minimum. That’s the recommended way. It does mean you need to set it lower then when the peak has passed.

It also works instead to just directly set the capacity outside the auto-scaling system, such as by using the CLI to update the table. This might cause the auto-scaling system to be a little confused however because you went around its back so it will at some point overwrite your change (which can be seen as good or bad, depending).