Azure aks agic identity refresh credentials

Question

I am deploying an aks cluster with agic (application gateway ingress controller) with bicep

In the aks.bicep I declare this

resource aksCluster 'Microsoft.ContainerService/managedClusters@2021-03-01' = {
  name: 'aks-core-${env}'
  location: resourceGroup().location
  identity: {
    type: 'SystemAssigned'
  }
  properties: {
    ...
    addonProfiles: {
      ...
      ingressApplicationGateway: {
        enabled: true
        config: {
          applicationGatewayId: applicationGatewayId
          effectiveApplicationGatewayId: applicationGatewayId
        }
      }
      
    }
  }
}

However for some reason the identity created by aks seems to need a role in the resource group created for the node pool, because an error "needs contributor role" appears,so I added this:

resource contributorRoleDefinition 'Microsoft.Authorization/roleDefinitions@2020-08-01-preview' existing = {
  scope: subscription()
  name: 'xxx'
}

resource aksfix 'Microsoft.Authorization/roleAssignments@2020-04-01-preview' = {
  name: guid(resourceGroup().id,'aksfix','Contributor')
  scope: resourceGroup()
  properties: {
    description: 'fixes aks cross resource group principal permissions for agic'
    principalId: aksCluster.properties.addonProfiles.ingressApplicationGateway.identity.objectId
    principalType: 'ServicePrincipal'
    roleDefinitionId: contributorRoleDefinition.id
  }
}

Now to be clear this WORKS, but not always. Even if the principal has the appropriate permissions, the same error keeps popping up for 30 min to 1-2 hours and then by magic it works!

I had a similar error when trying to use internal load balancers which requires network contributor to the kubelet principal, and the same behavior appears, it takes an insane amount of time to reflect the role changes, now what seems interesting is a message "if your permissions changed try refreshing your credentials" on the agic pod, is there a way to force a cluster to refresh its credentials (without creating a new service principal)?