Add scaling tests for Knative Eventing

[aslom]

Problem
There is no tests that could be run for all Knative Eventing components (sources, channels, brokers, …) and their compositions (sequences etc.) to determine scaling limits of Knative Eventing and experiment with autoscaling (with Knative Serving, HPA and KEDA autoscalers). The scaling tests should be easy to configure, run, and get results.

High level plan:

• - running tests directly to determine baseline HTTP performance by sending events from test driver to receiver (Knative sink) that computes metrics
• - running tests against Kafka directly to determine baseline Kafka broker performance by producing events as Kafka messages from test driver to receiver (Kakfa consumer) that computes metrics
• - running tests against Knative Kafka event source by sending events to Kafka and running receiver that computes metrics
• - run tests with KEDA v2 used to autoscale Knative Kafka sources to gather scaling metrics
• - run tests against IMC and Kafka channel
• - run tests against IMC and Kafka broker
• - attempt to use KEDA with channels and/or brokers and see how autosclaing works

Persona:
Which persona is this feature for?
System Integrator
Contributors

Exit Criteria
For given installation of Knative Eventing system integrators and/or contributors can run scaling tests with any workload simulated and get immediate results.

Time Estimate (optional):
How many developer-days do you think this may take to resolve?
3w

Additional context (optional)
The tests are to support discussing Proposal for Autoscaling of Knative Eventing

[aslom]

/assign

[aslom]

The first step is to explore options for Autoscaling of Knative Pull-based Event Sources by running tests against Knative Kafka event source: determine max supported load for particular config and then autoscaling with KEDA.

To determine scaling limits the test will run first against Kafka broker alone (Kafka topic with multiple partitions and Kafka consumers). Then against Knative Kafka event source with HTTP receiver as Knative sink to compare throughput and latency and what are scaling limits by using several setups of memory and CPU (to see how configurations affecting throughput and latency). Then test modified setup with KEDA ScaledObject for autoscaling to gather metrics about autoscaling performance for pull based event source.

[lberk]

@aslom Thanks for filing this issue! IIRC there were a variety of similar scaling issues, is there any way we could organize them so that we have a common entrypoint? (project/umbrella issue perhaps?) thanks!

[matzew]

@maschmid see here for "scaling" tests, like HPA ...

[github-actions]

This issue is stale because it has been open for 90 days with no
activity. It will automatically close after 30 more days of
inactivity. Reopen the issue with /reopen. Mark the issue as
fresh by adding the comment /remove-lifecycle stale.