Subnet is showing as destroyed in terraform destroy but it is not getting removed from azure portal

Question

I have written a module called network_resources in which I am creating vnets and subnets attached to it. Below is the code for the module and how the module is being called:

main.tf

resource "azurerm_virtual_network" "vnets" {
  for_each            = var.vnets
  name                = each.key
  resource_group_name = var.resource_group_name
  location            = var.location
  address_space       = [each.value.address_space]
  dns_servers         = each.value.dns_servers
}

resource "azurerm_subnet" "subnets" {
  for_each             = local.subnets
  name                 = each.value.subnet_name
  resource_group_name  = var.resource_group_name
  virtual_network_name = azurerm_virtual_network.vnets[each.value.vnet_name].name
  address_prefixes     = [each.value.subnet_address]
  service_endpoints    = each.value.service_endpoints
}

local.tf

locals {
  subnets_flatlist = flatten([for key, val in var.vnets : [
    for subnet in val.subnets : {
      vnet_name         = key
      subnet_name       = subnet.subnet_name
      subnet_address    = subnet.subnet_address
      service_endpoints = subnet.service_endpoints
    }
    ]
  ])
  subnets = { for subnet in local.subnets_flatlist : subnet.subnet_address => subnet }
}

variables.tf

variable "resource_group_name" {
  description = "Name of the resource group to be imported."
  type        = string
}

variable "location" {
  description = "The location of the vnet to create. Defaults to the location of the resource group."
  type        = string
  default     = null
}

variable "vnets" {
  type = map(object({
    address_space = string
    dns_servers = list(string)
    subnets = list(object({
      subnet_name    = string
      subnet_address = string
      service_endpoints = list(string)
    }))
  }))
}

Code to call the module:

module "network_aks_prod1" {
  source                = "./network_resources_dns"
  vnets                 = var.vnets_aks_prod1
  resource_group_name = azurerm_resource_group.rg2.name
  location            = azurerm_resource_group.rg2.location
}

Variables.tf

vnets_aks_prod1 = {
    "bupaanz-mel-prod-caas-vnet01" = {
      address_space = "10.80.0.0/18"
      #dns_servers         = ["10.0.0.4" , "10.0.0.5"]
      dns_servers         = ["10.0.0.6","10.0.0.5", "10.0.0.6", "10.0.0.4","10.64.150.11"]
      subnets = [
        {
          subnet_name    = "subnet-mel-prod-aks-mgmt-10.80.9.64"
          subnet_address = "10.80.9.64/26"
          service_endpoints = []
        },
        {
          subnet_name    = "subnet-mel-prod-aks-internal1-10.80.0.0"
          subnet_address = "10.80.0.0/22"
          service_endpoints = []          
        },
        {
          subnet_name    = "GatewaySubnet"
          subnet_address = "10.80.9.0/26"
          service_endpoints = []         
        },  
        {
          subnet_name    = "subnet-mel-prod-aks-internal2-10.80.4.0"
          subnet_address = "10.80.4.0/22"
          service_endpoints = []     
        },     
        {
          subnet_name    = "subnet-mel-prod-aks-pa1-ext-10.80.8.0"
          subnet_address = "10.80.8.0/25"
          service_endpoints = []     
        },
        {
          subnet_name    = "subnet-mel-prod-aks-pa2-int-10.80.8.128"
          subnet_address = "10.80.8.128/25"
          service_endpoints = []     
        },
      ]
    },
}

All the vnets and subnets get created successfully with the above code. Now if I have to delete one subnet, i am removing the code for one of the subnet with all its attributes. Now when I run the terraform plan and apply again, it informs that one subnet will be deleted. It is showing in terraform apply also the subnet has got deleted but it is not getting deleted in the portal. Please can you let me know how the subnet will get deleted in the portal with the terraform code only using the code which I am using.

Answer 1

Update as of 18th May 2022

Following further triage, it appears that azurerm_virtual_network resource is being modified because of a change of tags/edge_zone/bgp_community/flow_timeout_in_minutes attributes.

While deleting a subnet, terraform plan shows it is going to modify above attributes but under the hoods, it is updating the azurerm_virtual_network resource from the state file which includes the subnet azurerm_subnet just deleted.

Workaround to get it working with azurerm_virtual_network is to add lifecycle block as shown below..

resource "azurerm_virtual_network" "vnets" {
  for_each            = var.vnets
  name                = each.key
  resource_group_name = var.resource_group_name
  location            = var.location
  address_space       = [each.value.address_space]
  dns_servers         = each.value.dns_servers

  lifecycle {
    ignore_changes = [
      # one or more of the below attributes
      tags, edge_zone, bgp_community, flow_timeout_in_minutes        
    ]
  }
}

TBH, I'm not sure of the side-effects of the above attributes by including them into lifecycle block but it doesn't update the vnet resources when these gets changed.

However this is a bug on Terraform provider, you must create an issue there to get this fixed.

You are unable to delete the subnet because you haven't set subnet = [] in the azurerm_virtual_network resource. Below is what the documentation here says::

Since subnet can be configured both inline and via the separate azurerm_subnet resource, we have to explicitly set it to empty slice ([]) to remove it.

Also, the doc says below::

Terraform currently provides both a standalone Subnet resource, and allows for Subnets to be defined in-line within the Virtual Network resource. At this time you cannot use a Virtual Network with in-line Subnets in conjunction with any Subnet resources. Doing so will cause a conflict of Subnet configurations and will overwrite subnets.

Nevertheless, applying an empty slice gave even worse results as it wiped off all subnets. I think it's better you use dynamic-blocks to create subnets as part of vnet resource rather than using subnet specific resource.

subnet {
    name           = "subnet-mel-prod-aks-pa1-ext-10.80.8.0"
    address_prefix = "10.80.8.0/25"
  }
  /*
  subnet {
    name           = "subnet-mel-prod-aks-pa2-int-10.80.8.128"
    address_prefix = "10.80.8.128/25"
  }
  */
  ...

From the above snippet, when I commented on the subnet, it did delete the only subnet I commented out. I'm not sure whether there is a bug to fix azurerm_subnet as its behavior is very weird.