Skip to content

Commit

Permalink
Document how the test logging works (#334)
Browse files Browse the repository at this point in the history
  • Loading branch information
jonahgraham authored Sep 25, 2024
1 parent 2c1a0f9 commit 9a4cb9f
Show file tree
Hide file tree
Showing 2 changed files with 23 additions and 0 deletions.
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@ for more details, including how to setup a Windows machine with msys2 to run the
Pull Requests built using GitHub actions.
In the GitHub actions result you can examine test report and download the `test-logs` artifacts which are the verbose logs of each test that was run.
See [Integration Tests readme](https://github.com/eclipse-cdt-cloud/cdt-gdb-adapter/blob/main/src/integration-tests/README.md) for more details
## Debugging
Expand Down
22 changes: 22 additions & 0 deletions src/integration-tests/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,3 +118,25 @@ The _MSYS2 MinGW 64-bit_ shell can be integrated into VSCode.
```json
"terminal.integrated.defaultProfile.windows": "MSYS2 MinGW 64-bit",
```

### Test Logs and Test Reports

Running a test or series of tests can generate a lot of output.
By default, the adapter runs with [verbose on](https://github.com/eclipse-cdt-cloud/cdt-gdb-adapter/blob/590fea58cfb2ee4984d766cd1b2140738d3ff110/src/integration-tests/utils.ts#L206-L207) and those logs are saved to the `test-logs` directory in the root of the adapter.
Within `test-logs` each test has its own directory (derived from the `describe`d name), and under that is a directory for the set of parameters used to run the tests (see below), and finally under that is a directory for each test (derived from the `it` name).

For example running the [`can launch and hit a breakpoint` test](https://github.com/eclipse-cdt-cloud/cdt-gdb-adapter/blob/590fea58cfb2ee4984d766cd1b2140738d3ff110/src/integration-tests/launch.spec.ts#L42-L52) with the default settings places the log in `test-logs/launch/defaults/can launch and hit a breakpoint.log`

#### Test Parameters

All the tests are run repeatedly, but with different high level parameters, such as running all tests in remote and local debugging.
These parameters are [automatically prefixed](https://github.com/eclipse-cdt-cloud/cdt-gdb-adapter/blob/590fea58cfb2ee4984d766cd1b2140738d3ff110/src/integration-tests/utils.ts#L252-L275) onto the name of the test.

#### Test Reports

Running tests also saves the summaries in JUnit style, suitable for integrating with other tools like Jenkins, in the `test-reports` directory.

#### GitHub Actions

The test reports and test logs from all tests [are saved](https://github.com/eclipse-cdt-cloud/cdt-gdb-adapter/blob/93a7ce9721b2510af2350c94f4bfc773dd966a8a/.github/workflows/build-pr.yml#L31-L43) in the GitHub actions as artifacts.
This is useful to help diagnose why a test fails on GitHub actions, but passes on local development.

0 comments on commit 9a4cb9f

Please sign in to comment.