Skip to content

Commit

Permalink
tests/e2e: Skip tests after analysis
Browse files Browse the repository at this point in the history
Based in the test analysis done for 18 days of nighty tests in
#1831 (comment)
the only containerd test failures we saw were:
- `TestLibvirtCreatePeerPodAndCheckEnvVariableLogsWithImageOnly` - three times
- `TestLibvirtCreatePeerPodAndCheckWorkDirLogs` - four times
- `TestLibvirtCreatePeerPodAndCheckEnvVariableLogsWithDeploymentOnly` - twice

Although the chances of failure for each of these tests is < 25%, we want to reduce
the re-runs required, so if we skip these we should have more stable CI tests.
It should also be noted that most of the failures were seen on the packer built images.
This is probably just chance, but might indicate that the peer pod boot speed is related
and we should re-evaluate again once we can remove the packer podvm images.

Signed-off-by: stevenhorsman <steven@uk.ibm.com>
  • Loading branch information
stevenhorsman authored and mkulke committed Jan 7, 2025
1 parent cbbbc2f commit 73c0dac
Showing 1 changed file with 9 additions and 0 deletions.
9 changes: 9 additions & 0 deletions src/cloud-api-adaptor/test/e2e/libvirt_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -66,16 +66,25 @@ func TestLibvirtCreatePeerPodAndCheckUserLogs(t *testing.T) {
}

func TestLibvirtCreatePeerPodAndCheckWorkDirLogs(t *testing.T) {
// This test is causing issues on CI with instability, so skip until we can resolve this.
// See https://github.com/confidential-containers/cloud-api-adaptor/issues/1831
SkipTestOnCI(t)
assert := LibvirtAssert{}
DoTestCreatePeerPodAndCheckWorkDirLogs(t, testEnv, assert)
}

func TestLibvirtCreatePeerPodAndCheckEnvVariableLogsWithImageOnly(t *testing.T) {
// This test is causing issues on CI with instability, so skip until we can resolve this.
// See https://github.com/confidential-containers/cloud-api-adaptor/issues/1831
SkipTestOnCI(t)
assert := LibvirtAssert{}
DoTestCreatePeerPodAndCheckEnvVariableLogsWithImageOnly(t, testEnv, assert)
}

func TestLibvirtCreatePeerPodAndCheckEnvVariableLogsWithDeploymentOnly(t *testing.T) {
// This test is causing issues on CI with instability, so skip until we can resolve this.
// See https://github.com/confidential-containers/cloud-api-adaptor/issues/1831
SkipTestOnCI(t)
assert := LibvirtAssert{}
DoTestCreatePeerPodAndCheckEnvVariableLogsWithDeploymentOnly(t, testEnv, assert)
}
Expand Down

0 comments on commit 73c0dac

Please sign in to comment.