Inconsistent failure when downloading terraform modules in azure pipelines

Question

working with a simple terraform build pipeline and everything seems to be according to plan except I get a "failed to download modules" error halfway through my terraform init step

So the step runs fine at first and I am able to begin initializing and downloading modules:

and here's where things get dicey, it then errors out saying it can't download modules due to host key verification failures and "could not read from remote repository"

Errors

Error: Failed to download module
│ 
│ Could not download module "***_***_***_***_*******" (aks.tf:2) source code
│ from
│ "git::ssh://[email protected]/v3/myOrg/Terraform/repo":
│ error downloading
│ 'ssh://[email protected]/v3/***/Terraform/repo':
│ /usr/bin/git exited with 128: Cloning into
│ '.terraform/modules/***_***_***_***_cluster'...
│ Host key verification failed.
│ fatal: Could not read from remote repository.
│ 
│ Please make sure you have the correct access rights
│ and the repository exists.
│ 

What I've tried:

• I tried both dynamically inserting a PAT at runtime as well as adding the access token in the extra header of the url of terraform module git repos from the solution by Emmanuel Sciara in this page

Authenticating with Azure Repos git module sources in an Azure Pipelines build

• I added an installsshkeytask

steps:
  - task: DownloadSecureFile@1
    name: sshPrivateKey
    displayName: 'Download SSH Key'
    inputs:
      secureFile: 'pipelinekeys'
 
 
  - task: InstallSSHKey@0
    displayName: 'Install SSH Key'
    inputs:
      knownHostsEntry: '*'
      sshPublicKey: $(sshPublicKey)
      sshKeySecureFile: 'pipelinekeys'

• Manually adding the pipeline under pipeline permissions in Project Settings > Repositories > Repo > Settings > Security

How its consumed

I'm calling the init step from a template in another repo

steps:
  - task: Bash@3
    displayName: 'Terraform Init'
    env:
      ARM_CLIENT_ID: $(AZURE_CLIENT_ID)
      ARM_CLIENT_SECRET: $(AZURE_CLIENT_SECRET)
      ARM_SUBSCRIPTION_ID: $(AZURE_SUBSCRIPTION_ID)
      ARM_TENANT_ID: $(AZURE_TENANT_ID)
    inputs:
      targetType: 'inline'
      workingDirectory: $(System.DefaultWorkingDirectory)
      script: |
        set -euo pipefail
        
        echo "Initialize"
        terraform init \
            -input=false \
            -backend-config="resource_group_name=${storage_rg}" \
            -backend-config="storage_account_name=${storage_Account}" \
            -backend-config="container_name=${blob_container}" \
            -backend-config="key=${blob_name}" 

Any idea on what im missing? just need to be pointed in the right direction, not sure where the issue even is

Answer 1

I figured it out. It was a known host issue.

It didn’t work when I specified ‘*’ in the ssh install task on the pipeline

Had to run

ssh-keyscan -H -t rsa vs-ssh.visualstudio.com > $env:userprofile/.ssh/known_hosts

On my local machine then paste the contents into a variable in my pipeline and pass it through to the ssh install task

- task: InstallSSHKey@0
  displayName: ‘Install SSH Key’
  Inputs:
    KnownHostEntry: ‘$(put-variable-here)’
    SshPublicKey: ‘$(put-public-key-var-her)’
    SshKeySecureFile: ‘put-private-key-file-here’

https://dev.to/pwd9000/connect-terraform-to-azure-devops-git-repos-over-ssh-163c