Add support for private terraform repos

[jlk]

• terrascan version 1.4.0
• Operating System: MacOS

Description

A terrascan user posted on Accurics Community that when using terraform modules stored on Terraform Cloud, terrascan fails at downloading the private module.

While terraform will make use of the token stored in ~/.terraformrc as documented here, terrascan doesn't seem to know of this...yet.

What I Did

I cloned the aws tf module over to a private tf.io repo for testing, along with a basic main.tf that references it. Scanning that project dies as seen below:

$ terrascan scan . 
2021-03-26T16:25:10.465-0700	error	downloader/module-download.go:113	error while fetching available modules for module: app.terraform.io/widgetswest/vpc/aws, at registry: app.terraform.io
2021-03-26T16:25:10.465-0700	error	commons/load-dir.go:100	failed to download remote module "app.terraform.io/widgetswest/vpc/aws". error: 'error looking up module versions: 401 Unauthorized'
2021-03-26T16:25:10.465-0700	error	commons/load-dir.go:121	failed to build unified config. errors: <nil>: Failed to read module directory; Module directory  does not exist or cannot be read.

2021-03-26T16:25:10.465-0700	error	cli/run.go:107	scan run failed{error 26 0  failed to build terraform allResourcesConfig}

[jlk]

Looks like terraform cloud auth is done via bearer token, so just need to see if we can add support for that. Think I see how to do so, will tinker over the weekend.

[devang-gaur]

@jlk you mean private terraform repos ?

[MrMickS]

Is there any progress on this?

[jlk]

hey @MrMickS yep - a little longer than my "over the weekend" comment, sorry. :)

Wasn't quite as easy as I first thought - we can't just replace the http client, so I had to figure out some of terraform's data structures, plus I've got a few other things going on.

I've got something working POC late last week - adding in ability to read that from .terraformrc and doing testing, should have a PR in today or tomorrow.

[jlk]

@jlk you mean private terraform repos ?

doh - thanks @dev-gaur

[MrMickS]

That sounds great. Thanks very much.

[jlk]

@MrMickS - Just merged in support for this. if you're eager, you can try with the latest version of the master branch, otherwise we should be cutting a new release in the next week or two. It should Just Work - light mention in the docs at https://github.com/accurics/terrascan/blob/master/docs/usage.md#private-terraform-module-repositories

If there's an issue, let's address in a new GH issue (this was auto-closed when I merged the PR). Hope it meets your needs!

[MrMickS]

The downloading from TF Cloud works well. Thanks.