use --parallel to set Ginkgo's --procs flag

[xrstf]

This is a suggestion to how we could handle --parallel to control the concurrency in Ginkgo. Importantly, setting --parallel to a value larger than 1 does not set E2E_PARALLEL=true, as this would make the go-runner automatically set --p --skip=\[Serial\], which would skip a few important test cases.

Since we would now already control the E2E_EXTRA_GINKGO_ARGS variable, I also made it available on the CLI. This would allow someone to run --parallel --extra-ginkgo-args="--parallel=true --skip=...." to really only run the parallel tests.

It would work like this:

$ hydrophone --conformance --parallel 20
00:19:10 INF API endpoint: https://yadda.yadda.yadda:6443
00:19:10 INF Server version: version.Info{Major:"1", Minor:"29", GitVersion:"v1.29.2", GitCommit:"4b8e819355d791d96b7e9d9efe4cbafae2311c88", GitTreeState:"clean", BuildDate:"2024-02-14T10:32:40Z", GoVersion:"go1.21.7", Compiler:"gc", Platform:"linux/amd64"}
00:19:10 INF Using namespace: conformance
00:19:10 INF Using conformance image: registry.k8s.io/conformance:v1.29.2
00:19:10 INF Using busybox image: registry.k8s.io/e2e-test-images/busybox:1.36.1-1
00:19:10 INF Test framework will start 20 thread(s) and use verbosity level 4.
00:19:10 INF Created Namespace conformance.
00:19:10 INF Created ServiceAccount conformance-serviceaccount.
00:19:10 INF Created Clusterrole conformance-serviceaccount.
00:19:10 INF Created ClusterRoleBinding conformance-serviceaccount-role.
00:19:10 INF Created Pod e2e-conformance-test.
⢆⡱2024/04/03 22:19:11 Running command:
Command env: []
Run from directory:
Executable path: /usr/local/bin/ginkgo
Args (comma-delimited): /usr/local/bin/ginkgo,--focus=\[Conformance\],--skip=,--no-color=true,--procs=20,--timeout=24h,/usr/local/bin/e2e.test,--,--disable-log-dump,--repo-root=/kubernetes,--provider=skeleton,--report-dir=/tmp/results,--kubeconfig=
2024/04/03 22:34:16 Now listening for interrupts
⢀⡱Running Suite: Kubernetes e2e suite - /usr/local/bin
====================================================
Random Seed: 1712183656 - will randomize all specs

Will run 388 of 7407 specs
Running in parallel across 20 processes

and dramatically speeds up the conformance tests, from ~1h30min to 20min.

I'm not sure if this is the best/correct solution, but it seems to work :-) Maybe the way --parallel here does not map to --parallel in Ginkgo is confusing, maybe it would make more sense to introduce a new flag / rename --parallel to something like --concurrency=<int>? Or even directly adapt Ginkgo's --procs/--nodes?

Fixes #167

[k8s-ci-robot]

Hi @xrstf. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[reetasingh]

removing L94-97 does not seem right, do we not need this?

[xrstf]

I didn't remove those lines, I moved them from this function into SetDefaults, where all the other status printing happens. Seems more natural to have all the validation in one func and all the information printing in the other.

[reetasingh]

/ok-to-test

[dims]

/assign @reetasingh @rjsadow

[rjsadow]

/lgtm
/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: rjsadow, xrstf

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [rjsadow]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment