Skip to content

Commit

Permalink
Backport of Add documentation for JUnit output from 'terraform test' …
Browse files Browse the repository at this point in the history
…into v1.11 (#36388)

* backport of commit 00eb798

* backport of commit e685e91

* backport of commit d8a2323

* backport of commit 97789c6

* backport of commit 1cb62b5

* backport of commit 5510c11

---------

Co-authored-by: Sarah French <sarah.french@hashicorp.com>
Co-authored-by: Sarah French <15078782+SarahFrench@users.noreply.github.com>
  • Loading branch information
3 people authored Jan 22, 2025
1 parent dba8e40 commit 2a3b546
Showing 1 changed file with 91 additions and 0 deletions.
91 changes: 91 additions & 0 deletions website/docs/cli/commands/test.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,8 @@ The following options apply to the Terraform `terraform test` command:

* `-json` - Displays machine-readable JSON output for your testing results.

* `-junit-xml=<output file path>` - Saves a test report in JUnit XML format to the specified file. This is currently incompatible with remote test execution using the the `-cloud-run` option. The file path must be relative or absolute.

* `-test-directory=<relative directory>` - Overrides the directory that Terraform looks into for test files. Note that Terraform always loads testing files within the main configuration directory. The default testing directory is `tests`.

* `-verbose` - Prints out the plan or state for each `run` block within a test file, based on the `command` attribute of each `run` block.
Expand Down Expand Up @@ -134,3 +136,92 @@ The testing directory must be beneath the main configuration directory, but it c
> Note: Test files within the root configuration directory are always loaded, regardless of the `-test-directory` value.
We do not recommend changing the default test directory. The option for customization is included for configuration authors who may have included a `tests` submodule in their configuration before the `terraform test` command was released. In general, the default test directory of `tests` should always be used.


## Example: Test Output Format Options

Below is a contrived example of Terraform testing that makes assertions about the values of local variables `true` and `false`.
There are two test files: one contains a passing test, and one contains a failing test.

```hcl
# main.tf
locals {
true = "true"
false = "true" # incorrect, should be "false"!
}
```

The assertion that `local.true == "true"` in `example_1.tftest.hcl` will pass:

```hcl
# example_1.tftest.hcl
run "true_is_true" {
assert {
condition = local.true == "true"
error_message = "local.true did not match expected value"
}
}
```

The assertion that `local.false == "false"` in `example_2.tftest.hcl` will fail:

```hcl
# example_2.tftest.hcl
run "false_is_false" {
assert {
condition = local.false == "false"
error_message = "local.false did not match expected value"
}
}
```

### Test output in JUnit XML format, saved to file

Below is the output of the `terraform test -junit-xml=./output.xml` command using the example files above.
The test output is:

* Printed to the terminal in the default, human-readable format.
* Also saved in JUnit XML format in the file specified by the flag.

Below is the contents of the resulting `output.xml` file:

```xml
<?xml version="1.0" encoding="UTF-8"?><testsuites>
<testsuite name="example_1.tftest.hcl" tests="1" skipped="0" failures="0" errors="0">
<testcase name="true_is_true" classname="example_1.tftest.hcl" time="0.002295" timestamp="2025-01-13T19:23:16Z"></testcase>
</testsuite>
<testsuite name="example_2.tftest.hcl" tests="1" skipped="0" failures="1" errors="0">
<testcase name="false_is_false" classname="example_2.tftest.hcl" time="0.001468" timestamp="2025-01-13T19:23:16Z">
<failure message="local.false did not match expected value"><![CDATA[
Error: Test assertion failed
on example_2.tftest.hcl line 3, in run "false_is_false":
3: condition = local.false == "false"
├────────────────
│ local.false is "true"
local.false did not match expected value
]]></failure>
</testcase>
</testsuite>
</testsuites>
```

If a run block contains a failing assertion, the `<testcase>` element will contain a `<failure>` element that includes the error message and further details.

A `<testcase>` element could contain a `<skipped>` element that includes details about why a test was skipped. This could either be due to an error causing remaining run blocks to be skipped, or due to the command being interrupted.

#### Mapping Terraform test command concepts to JUnit XML format

The test report generated when using `-junit-xml` maps Terraform test command concepts to JUnit XML format according to the table below:

| Terraform test concept | Element in JUnit XML output |
| ----------------------------------| --------------------------------------------|
| Test directory | `<testsuites>` |
| Test file | `<testsuite>` |
| Run block | `<testcase>` |
| Run block assertion | None; details are included only on failure |
| Test failure | `<failure>` |
| Test was skipped | `<skipped>` |
| Test stopped due to error | `<error>` |
| Any unhandled warnings or errors | `<system-err>` |

0 comments on commit 2a3b546

Please sign in to comment.