Conformance validation

[mayankshah1607]

View rendered

Conformance validation

This RFC proposes a new test suite that shall be used for conformance validation. The new test suite shall be used to validate non-trivial network communication (HTTP, gRPC, websocket) among stateless and stateful workloads in the Linkerd data plane. This shall be done by carrying out extensive e2e tests of Linkerd features using a sample distributed application (data plane).

Problem Statement

Linkerd has an extensive check suite that validates a cluster is ready to install Linkerd and that the installation was successful. These checks are, unfortunately, static checks. Because of the wide number of ways a Kubernetes cluster can be configured, users want a way to validate their specific install for conformance over time. The proposed project tackles this problem by allowing users to deploy sample workloads to their cluster and carry out extensive E2E tests for conformance.

Signed-off-by: Mayank Shah mayankshah1614@gmail.com

[mayankshah1607]

Currently, the RFC proposes that users interact with the testing tool either using a shell script, or using Sonobuoy via its plugin model. We may want to consider how comfortable users will be with using Sonobuoy before having to depend on it. Here is an example of a tool that uses Sonobuoy's plugin model for conformance testing. Although our use-cases are different, I'd like to point out the UX and the way users shall interact with our Sonobuoy plugin would remain somewhat similar.
To keep things less complicated, it would be nice to have the proposed shell script that kicks off our conformance tests by building a go test command by accepting a group of flags. However, we may then have to consider two things:

• Would a user then be comfortable with cloning the linkerd2 repo for the sake of running the tests? If that's an issue -
• is it a good idea to house these tests in a separate repo?

Further, the original issue linkerd/linkerd2#1096 talks about kicking off these tests via the CLI using a command like linkerd conformance. In that case, how would the test suite be designed? For e.g, instead of having a suite of _test.go files, would it be nice to draw ideas from the way the check suite is designed?

[grampelberg]

@Pothulapati mind taking a pass on this?

[olix0r]

This was accidentally closed when we changed the default branch to main. The base should probably be updated...

Is there more to do on this RFC?