Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Validate doesn't work for projects with nested sub-directories #341

Closed
jamiezieziula opened this issue Feb 10, 2022 · 9 comments
Closed
Labels

Comments

@jamiezieziula
Copy link

What problem are you facing?

When running pre-commit run --all-files or when its run automatically on a commit, i get the following result, even when some terraform code is note valid:

Terraform fmt............................................................Passed
Terraform validate.......................................................Passed
Terraform docs...........................................................Passed

My terraform code is nested in various folders throughout my repo, but in this specific case i have the following folder structure:

.
├── LICENSE
├── README.md
├── aws
│   ├── README.md
│   ├── aws-services
│   │   ├── delete-default-vpcs
│   │   │   ├── README.md
│   │   │   ├── data.tf
│   │   │   ├── iam.tf
│   │   │   ├── main.py
│   │   │   ├── main.tf
│   │   │   ├── providers.tf
│   │   │   ├── variables.tf
│   │   │   └── versions.tf
│   │   ├── network
│   │   │   ├── README.md
│   │   │   ├── data.tf
│   │   │   ├── main.tf
│   │   │   ├── outputs.tf
│   │   │   ├── providers.tf
│   │   │   ├── variables.tf
│   │   │   └── versions.tf
│   │   └── state-management
│   │       ├── README.md
│   │       ├── main.tf
│   │       ├── providers.tf
│   │       ├── variables.tf
│   │       └── versions.tf
└── setup.cfg

How could pre-commit-terraform help solve your problem?

Reviewing this PR, it appears that this hook should work, but for some reason, I can't replicate

@jamiezieziula jamiezieziula added the feature New feature or request label Feb 10, 2022
@yermulnik yermulnik changed the title Validate doesn't work for projects with nexted sub-directories Validate doesn't work for projects with nested sub-directories Feb 11, 2022
@MaxymVlasov
Copy link
Collaborator

What hook version do you use?
In v1.64.0 it works fine

.pre-commit-config.yaml:

repos:
- repo: https://github.com/antonbabenko/pre-commit-terraform
  rev: v1.64.0
  hooks:
    - id: terraform_validate
      args:
        - --init-args=-upgrade
        - --init-args=-get=true
        - --envs=AWS_DEFAULT_REGION="us-west-2"
        - --envs=AWS_ACCESS_KEY_ID="anaccesskey"
        - --envs=AWS_SECRET_ACCESS_KEY="asecretkey"

tree -d:

.
├── environment
│   ├── prd
│   └── qa
├── modules
│   ├── aws-acm-certificate
│   ├── aws-api-gateway
│   ├── aws-chatbot
│   ├── aws-dynamodb-table
│   ├── aws-ecr-repo
│   ├── aws-eks-cluster
│   ├── aws-eks-node-group
│   │   └── user_data
│   ├── aws-elasticache-cluster
│   ├── aws-environment
│   │   ├── templates
│   │   └── user_data
│   ├── aws-gitlab-ec2-manager
│   │   └── templates
│   ├── aws-grafana
│   ├── aws-iam-group
│   ├── aws-iam-role
│   ├── aws-iam-user
│   ├── aws-lambda-function
│   ├── aws-msk-cluster
│   ├── aws-rds-cluster
│   ├── aws-route53-zone
│   ├── aws-s3-bucket
│   ├── aws-sns-topic
│   ├── helm-release
│   ├── kube-manifest
│   ├── postgresql-env
│   └── rapp-services
│       ├── aws-core-api
│       ├── aws-iot-segway-max-plus
│       ├── aws-iot-segway-mock
│       └── aws-ops-web
├── path
├── services
│   ├── alerting
│   │   └── us-east-1
│   ├── chatbot
│   │   └── global
│   ├── dns
│   │   └── global
│   ├── docker-registry
│   │   └── global
│   ├── grafana
│   │   └── qa
│   ├── iam
│   │   └── global
│   ├── kafka
│   │   ├── prd
│   │   └── qa
│   ├── kubernetes
│   │   ├── prd
│   │   └── qa
│   ├── postgresql
│   │   └── qa
│   ├── rapp
│   │   ├── prd
│   │   │   └── p-1
│   │   │       ├── core-api
│   │   │       ├── iot-segway-max-plus
│   │   │       └── ops-web
│   │   └── qa
│   │       ├── commons
│   │       ├── qa-1
│   │       ├── qa-2
│   │       ├── qa-3
│   │       └── stg-1
│   ├── redis
│   │   ├── prd
│   │   └── qa
│   └── s3
│       ├── global
│       │   └── backend
│       └── qa
└── test

Feel free to reopen issue if will not works with latest version. And provide details specified in bug report template

@MaxymVlasov MaxymVlasov added hook/terraform_validate Bash hook and removed feature New feature or request labels Feb 11, 2022
@yermulnik
Copy link
Collaborator

Seems like I'm having sort of the same behavior as reporter (or I might be misinterpreting how this hook should work 🤔):

> tree broken/
broken/
├── test.tf
└── tf
    ├── code
    │   └── test.tf
    └── test.tf

2 directories, 3 files

> find broken/ -type f | while read F; do ls $F && cat $F && echo ====; done
broken/test.tf
test
====
broken/tf/test.tf
test
====
broken/tf/code/test.tf
test
====

> cat .pre-commit-config.yaml | egrep -v "^#"
repos:
- repo: https://github.com/antonbabenko/pre-commit-terraform
  #rev: f3dd6deac411ee9144a22752db7a53923685fbce
  rev: v1.64.0
  hooks:
    - id: terraform_validate

> pre-commit clean
Cleaned /home/giermulnik/.cache/pre-commit.

> pre-commit run -a
[INFO] Initializing environment for https://github.com/antonbabenko/pre-commit-terraform.
Terraform validate.......................................................Passed

Obviously I'd expect terraform validate to error out on each of the files in nested dirs like this:

> cp broken/test.tf ./test.tf
> pre-commit run -a
Terraform validate.......................................................Failed
- hook id: terraform_validate
- exit code: 1

Validation failed: .
╷
│ Error: Argument or block definition required
│
│   on test.tf line 1:
│    1: test
│
│ An argument or block definition is required here. To set an argument, use
│ the equals sign "=" to introduce the argument value.
╵

@MaxymVlasov
Copy link
Collaborator

16:24 341 git:(main +)
➜ git status
On branch main

No commits yet

Changes to be committed:
  (use "git rm --cached <file>..." to unstage)
        new file:   .pre-commit-config.yaml
        new file:   broken/test.tf
        new file:   broken/tf/code/test.tf
        new file:   broken/tf/test.tf


16:24 341 git:(main +)
➜ tree
.
└── broken
    ├── test.tf
    └── tf
        ├── code
        │   └── test.tf
        └── test.tf

3 directories, 3 files

16:24 341 git:(main +)
➜ cat .pre-commit-config.yaml
repos:
- repo: https://github.com/antonbabenko/pre-commit-terraform
  rev: v1.64.0
  hooks:
    - id: terraform_validate
      args:
        - --init-args=-upgrade
        - --init-args=-get=true
        - --envs=AWS_DEFAULT_REGION="us-west-2"
        - --envs=AWS_ACCESS_KEY_ID="anaccesskey"
        - --envs=AWS_SECRET_ACCESS_KEY="asecretkey"


16:24 341 git:(main +)
➜ pre-commit run -a
Terraform validate.......................................................Failed
- hook id: terraform_validate
- exit code: 1

Init before validation failed: broken
There are some problems with the configuration, described below.

The Terraform configuration must be valid before initialization so that
Terraform can determine which modules and providers need to be installed.

Error: Argument or block definition required

  on test.tf line 1:
   1: fsdsdf

An argument or block definition is required here. To set an argument, use the
equals sign "=" to introduce the argument value.

Init before validation failed: broken/tf
There are some problems with the configuration, described below.

The Terraform configuration must be valid before initialization so that
Terraform can determine which modules and providers need to be installed.

Error: Argument or block definition required

  on test.tf line 1:
   1: fsdsdf

An argument or block definition is required here. To set an argument, use the
equals sign "=" to introduce the argument value.

Init before validation failed: broken/tf/code
There are some problems with the configuration, described below.

The Terraform configuration must be valid before initialization so that
Terraform can determine which modules and providers need to be installed.

Error: Argument or block definition required

  on test.tf line 1:
   1: fsdsdf

An argument or block definition is required here. To set an argument, use the
equals sign "=" to introduce the argument value.

$ find broken/ -type f | while read F; do ls $F && cat $F && echo ====; done
broken/tf/code/test.tf
fsdsdf
====
broken/tf/test.tf
fsdsdf
====
broken/test.tf
fsdsdf
====

@yermulnik
Copy link
Collaborator

yermulnik commented Feb 16, 2022

Weird. This just doesn't behave like on your end:

> ls -la
total 16
drwx------ 2 giermulnik giermulnik  4096 Feb 16 16:50 .
drwx------ 6 giermulnik giermulnik 12288 Feb 16 16:44 ..

> git init
hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint:
hint:   git config --global init.defaultBranch <name>
hint:
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint:
hint:   git branch -m <name>
Initialised empty Git repository in /home/giermulnik/tmp/www/.git/

> cat <<EOF>.pre-commit-config.yaml
continue> repos:
continue> - repo: https://github.com/antonbabenko/pre-commit-terraform
continue>   rev: v1.64.0
continue>   hooks:
continue>     - id: terraform_validate
continue> EOF

> wget -q https://raw.githubusercontent.com/antonbabenko/pre-commit-terraform/master/.pre-commit-hooks.yaml

> mkdir -p broken/tf/code/

> for DIR in broken/tf/code broken/tf broken; do echo fsdsdf > $DIR/test.tf; done

> tree -a -I ".git"
.
├── broken
│   ├── test.tf
│   └── tf
│       ├── code
│       │   └── test.tf
│       └── test.tf
├── .pre-commit-config.yaml
└── .pre-commit-hooks.yaml

3 directories, 5 files

> pre-commit clean
Cleaned /home/giermulnik/.cache/pre-commit.

> pre-commit gc
0 repo(s) removed.

> pre-commit autoupdate
Updating https://github.com/antonbabenko/pre-commit-terraform ... already up to date.

> pre-commit run -a -v
[INFO] Initializing environment for https://github.com/antonbabenko/pre-commit-terraform.
Terraform validate...................................(no files to check)Skipped
- hook id: terraform_validate

@MaxymVlasov
Copy link
Collaborator

Did you run git add -A before pre-commit run -a?

@yermulnik
Copy link
Collaborator

yermulnik commented Feb 16, 2022

Oh my god 🤦🏻 The most important step was missing from my setup 🤦🏻 Seems like topicstarter most probably has the same "issue".

@MaxymVlasov
Copy link
Collaborator

pre-commit can't check files that are not already added to git :)

@jamiezieziula
Copy link
Author

jamiezieziula commented Feb 16, 2022 via email

@yermulnik
Copy link
Collaborator

yermulnik commented Feb 16, 2022

@jamiedick Please provide further info since what you provided initially didn't have any detailed info: show us git status of your repo, run terraform validate on "broken" file and show us result, cleanup pre-commit's cache (clean and gc), autoupdate pre-commit, show pre-commit config file from your repo, etc (you may get more insight looking at what Max (and I, though unintentionally wrong) did up above).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants