[WIP] Experimental KEDA support for Kafka Event Source

[aslom]

Warning: this is experimental and may be changed in future. Should not be used in production. This is mainly for discussion and evolving scaling in Knative eventing.

The code is using Unstructured and also imported KEDA API - this is for discussion which version should be used (right now only Unstructured is fully implemented).
KEDA to provide client-go support discussion #494 kedacore/keda#494

Release Note

Experimental KEDA support for Kafka Event Source

[googlebot]

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors. If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and then comment @googlebot I fixed it.. If the bot doesn't comment, it means it doesn't think anything has changed.

ℹ️ Googlers: Go here for more info.

[googlebot]

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors. If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and then comment @googlebot I fixed it.. If the bot doesn't comment, it means it doesn't think anything has changed.

ℹ️ Googlers: Go here for more info.

[lberk]

I assume we'll also want to include a finalizer, or some mechanism to ensure we delete the scaled objects we created/own as we delete the kafkasource itself?

[aslom]

@lberk for cleanup using OwnerReferences should allow to automatically delete ScaledObject if owner of it is KafkaSource without need to do finalizer?

[zroubalik]

@lberk for cleanup using OwnerReferences should allow to automatically delete ScaledObject if owner of it is KafkaSource without need to do finalizer?

yeah, it should be like that

[aslom]

@googlebot I fixed it.

[mattmoor]

Produced via:
prettier --write --prose-wrap=always $(find -name '*.md' | grep -v vendor | grep -v .github | grep -v docs/cmd/)

[mattmoor]

Produced via:
prettier --write --prose-wrap=always $(find -name '*.md' | grep -v vendor | grep -v .github | grep -v docs/cmd/)

[knative-prow-robot]

@aslom: The following test failed, say /retest to rerun all failed tests:

Test name	Commit	Details	Rerun command
pull-knative-eventing-contrib-build-tests	9339051	link	/test pull-knative-eventing-contrib-build-tests

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[knative-prow-robot]

@aslom: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[matzew]

Is this in sync with the work that @zroubalik is proposing?

Or should we close this ?

[aslom]

@matzew I am checking with @zroubalik

[zroubalik]

I am using a separate controller and not using this stuff. I'd like to present the initial work on the next week WG or the week after (I have a conflict that time, so it's not that easy for me to join).
I'd keep this open until I show my proposal and then we can choose which direction we would like to go.

[aslom]

I have chatted with Zbynek and it seems his approach is more general and may work better. After he presented it to WG then I am going to close this issue.

[duglin]

cool - looking forward to seeing it.

[matzew]

@aslom with the new doc you presented in the WG, should we close this now ?

[zroubalik]

On the latest WG I have proposed this approach: https://github.com/zroubalik/autoscaler-keda

I think we can close this one.

[aslom]

Yes. Closing it for ow as we explore other options: knative/eventing#2901 (comment)