Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

conformance validation: Add new helper method in testutil package #4530

Closed
mayankshah1607 opened this issue Jun 2, 2020 · 1 comment · Fixed by #4532
Closed

conformance validation: Add new helper method in testutil package #4530

mayankshah1607 opened this issue Jun 2, 2020 · 1 comment · Fixed by #4532

Comments

@mayankshah1607
Copy link
Contributor

mayankshah1607 commented Jun 2, 2020

Problem

The testutil package offers a set of utility functions and helpers for the integration tests. As a part of the Conformance Validation project (RFC), we plan on reusing these utility functions to avoid having to reinvent the wheel or make things redundant by re-writing the same helpers in a different package. How this can be done is by calling the testutil.NewTestHelper() method which returns a new instance of testutil.TestHelper, on which we call the utility functions.

However, the problem with this approach is that the testutil.NewTestHelper() requires you to pass a set of flags that are only useful to the integration tests. For example, the -integration-tests flag, which determines if the integration tests should be run, errors out if it is not set to true. The -linkerd flag, which determines the path of the linkerd2 binary, won't serve any purpose in the conformance tests, as we plan on using Sonbuoy, which pulls up a pod, that will always look at a fixed file path that points to the binary of a particular version which will be mentioned in the Sonobuoy plugin.
The -linkerd-namespace flag, which specifies the namespace of the control plane, is by default set to l5d-integration. Moreover, some of the other flags are irrelevant to conformance testing. The Conformance Validation project would most certainly benefit from reusing the testutil package without having to depend on these rules or flag options configured specifically for the integration tests.

How should this be solved?

We propose to have a new helper method testutil.NewConformanceTestHelper which is similar to testutil.NewTestHelper - it returns a new instance of testutil.TestHelper, but instead of depending on flag options, the values of the fields in testutil.TestHelper shall be passed as function arguments. This way, not only will we have full control over the configuration options from the conformance testing side while being able to reuse the utility functions from testutil package, but will also enhance UX by hiding away irrelevant flag options. This new method shall be used by the linkerd2-conformance repo. This will also ensure that the existing integration test workflow does not break.

cc @Pothulapati @grampelberg

@mayankshah1607
Copy link
Contributor Author

I do not see any use case for this new helper method apart from conformance validation, hence the name NewConformanceTestHelper. But if we do plan on taking this approach, we could use general naming conventions like NewBaseHelper, or something alike.

mayankshah1607 added a commit to mayankshah1607/linkerd2 that referenced this issue Jun 2, 2020
- Add new struct `ConformanceTestHelperOptions`

`ConformanceTestHelperOptions` defines the TestHelper
options required for conformance validation - https://github.com/linkerd/linkerd2-conformance
This way the default options may be defined in the linkerd2-conformance repo,
and an instance can be passed to NewConformanceTestHelper() to obtain a *TestHelper

- Add new helper function `NewConformanceTestHelper`

`NewConformanceTestHelper` accepts a *ConformanceTestHelperOptions and
returns a new *TestHelper from the options provided. This helper was
created to be able to reuse the `testutil` package for Conformance validation

See - linkerd#4530

Signed-off-by: Mayank Shah <mayankshah1614@gmail.com>
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jul 17, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant