Terraform layout for multi-client multi-tenant multi-environment using battle-tested versioned modules

Question

I am trying to write Terraform code for a greenfield Azure project and below are the requirements:

• The project is multi-client - Client A, Client B, Client C ... future Clients
• Each client will have multiple environment Prod(Tenant1),Staging (Tenant2) and Non-Prod (Dev,QA,UAT,Pre-staging) (Tenant3)
• DRY code using versioned modules hosted within Gitlab internally
• TFState file has to be stored in Azure Storage Account - safe, encrypted and with locking mechanism.
• Deployment has to be done using GitLab CI/CD

I need some best practices advice and examples and answers to the following questions:

1. What will be the folder layout for the modules (child modules) repository? with example versioned module and variable definition?
2. What will be the layout of the project (root modules) assuming that I want to isolate each client from the other client and each environment from the other environment?
3. Do we use .tfvars or Terraform Workspaces to achieve the DRY code? or Terragrunt assuming we are using Terraform >0.12?. If we use .tfvars, how do we hide the sensitive information (Terraform <0.13) [Terraform >0.14 has sensitive: true]
4. What will be the design of the Gitlab CI/CD pipeline? I can imagine the pipeline stages as: dev-tf-plan --> dev-tf-apply --> qa-tf-plan --> qa-tf-apply --> uat-tf-plan --> uat-tf-apply ....... prod-tf-plan --> prod-tf-apply This we a single branch (master) can progress through all the environments How can we expand this to multiple-client?
5. Do keep all the modules in a single git repository or keep each sub-module in its own git repository?

Please share your ideas and experiences in working on similar scenarios.

Thanks

Answer 1

yes, you need terragrunt, the terraform wrapper, it covers all of your idea, without too much you need care.

Go through this sample and start with it:

https://github.com/gruntwork-io/terragrunt-infrastructure-live-example

add the folder structures for your environments,

client1
  | _global
  |-dev
    |-us-east-1
  |-prod
    |-us-east-2
client2
  | _global
  |-dev
  |prod