Terraform module in github private repo

Question

I am trying to use my private git repo as source for the terraform modules.

ssh public key has been copied over to github.

Tried following options as source but nothing worked:

• git::[email protected]:/terraform-modules//sub-dir
• github.com/terraform-modules//sub-dir
• [email protected]:/terraform-modules//sub-dir
• git::https://MACHINE-USER:[email protected]/terraform-modules//sub-dir
• git::https://github.com//terraform-modules.git//subdir
• https://[email protected]/terraform-modules.git//subdir

Any help is greatly appreciated.

Referring to Private Github repos section on the following link didn't help either. https://github.com/alibaba/terraform-provider/blob/master/vendor/github.com/hashicorp/terraform/website/docs/modules/sources.html.markdown

Private GitHub Repos If you need Terraform to fetch modules from private GitHub repos, you must provide Terraform with credentials to authenticate as a user with read access to those repos.

If you run Terraform only on your local machine, you can specify the module source as an SSH URI (like [email protected]:hashicorp/example.git) and Terraform will use your default SSH key to authenticate.

If you use Terraform Enterprise, you can use SSH URIs. You'll need to add an SSH private key to your organization and assign it to any workspace that fetches modules from private repos. See the Terraform Enterprise docs about SSH keys for cloning modules.

If you need to run Terraform on a remote machine like a CI worker, you either need to write an SSH key to disk and set the GIT_SSH_COMMAND environment variable appropriately during the worker's provisioning process, or create a GitHub machine user with read access to the repos in question and embed its credentials into the modules' source parameters: module "private-infra" { source = "git::https://MACHINE-USER:[email protected]/org/privatemodules//modules/foo" } Note that Terraform does not support interpolations in the source parameter of a module, so you must hardcode the machine username and password if using this method.

Answer 1

This worked for me

module "name_of_module" {
  source = "git::https://<user>:<pat>@github.com/folder/terraform-azure-core-resource-group.git"
  ...
}

Answer 2

This worked for me:

1. Set up your ssh keys; make sure that your ~/.ssh/config file has a block like this:

Host USERNAME.github.com
    HostName github.com
    User git
    IdentityFile ~/.ssh/id_rsa 

2. Add this to your .tf file:

module "name_of_module" {
  source = "[email protected]:USERNAME/REPONAME.git//SUBDIR"
  ...
}

Answer 3

Things needed:

• A GitHub machine account (Note: This is not much different from a regular GitHub account functionally; it is referred to as "machine" based on the intended usage). See machine users.
• An ssh key. Note: I used RSA. Example on how to generate one:

$ ssh-keygen -t rsa -b 4096 -C "[email protected]"

• Associate the public key with the GitHub machine account. See adding-a-new-ssh-key-to-your-github-account.
• Then, either in your particular repo OR as an organizational secret, add the private ssh key in GitHub.

Note: If you configure the organizational secret to be available to specific repos, be sure to specify the repo that has the Terraform code that you are attempting to import.

• Then, in your GitHub Action yaml file, add the code that adds the private ssh key to the runner's ssh agent, to be able to clone the Terraform module that is in a private GitHub repo. Example:

      - name: Terraform Init
        id: init
        run: terraform init
        env:
          GIT_SSH_COMMAND: "echo '${{ secrets.ORG_PRIVATE_SSH_KEY }}' > id_rsa
          && ssh-keyscan github.com > known_hosts
          && chmod 600 id_rsa known_hosts
          && ssh -i ./id_rsa -o UserKnownHostsFile=./known_hosts"

      - name: Terraform Plan
        id: plan
        if: github.event_name == 'pull_request'
        run: terraform plan -no-color
        continue-on-error: true
        env:
          GIT_SSH_COMMAND: "echo '${{ secrets.ORG_PRIVATE_SSH_KEY }}' > id_rsa
          && ssh-keyscan github.com > known_hosts
          && chmod 600 id_rsa known_hosts
          && ssh -i ./id_rsa -o UserKnownHostsFile=./known_hosts"

Reference/Credit: https://github.com/hashicorp/setup-terraform/issues/33

Note: There appears to be many ways to do such things when googling but I labored over this for weeks trying the various options and ultimately was able to do it with this AND I understood how it worked. :) I encourage feedback.

Answer 4

Tested on bitbucket. It should be the same on github:

source = "git::https://@bitbucket.com/mycompany/my-project.git"

Answer 5

in our organisation we were also using terrafom modules created in the private git repos and it worked locally but in the github action it was real pain to resolve. here is how i resolve it. You need user account that is member of the repo and has the git access token associated with it and change your workflow plan file as below:

jobs:
  plan:
    environment: production
    runs-on: ubuntu-latest
    name: New Relic terraform plan
    steps:
      - name: Checkout
        uses: actions/checkout@v2
      - name: terraform plan
        uses: dflook/[email protected]
        env:
          TERRAFORM_HTTP_CREDENTIALS: github.com/DigitalInnovation=<userName>:${{ secrets.GIT_TOKEN }}
    

Answer 6

I will not go for SSH, and none of the above work for me in the CI for private repository

git config --global url."https://${USER}:${USER_READONLY_TOKEN}@github.com/my_orgnization".insteadOf "https://github.com/my_orgnization"

Make sure the token is read-only privileges and organizations my_orgnization name is correct

      - run:
          name: remote backend
          command: |
            git config --global url."https://${USER}:${USER_READONLY_TOKEN}@github.com/my_orgnization".insteadOf "https://github.com/my_orgnization"
            terraform init