KedaSource duck type

[nachocano]

• Created a new v1alpha1 duck called KedaSource, which inherits v1.Source and has some specific KEDA configuration options. This is related to discussions that started sometime back in the Sources WG to support scalable Pull-based Sources, and is based on the awesome work started by @aslom.

• Cosmetic: upstreamed some serving filter utilities that will be needed by other repos. For example, for doing annotation-based filtering in reconcilers, etc. This can actually be done in a separate PR if folks prefer, as it might add noise.

Some thoughts:

• Although not optimal, I consider pkg a better place to put this, instead of eventing. Similar rationale as with the Sources duck, folks might be creating KEDA Sources without depending on eventing.

• Decided to use a separate v1alpha1 duck for KEDA and not "pollute" the v1.Source we have. Some reasons are:

  a) this is still in an initial phase (as well as KEDA),
  b) I prefer more typed things, i.e., having a generic "typed" scaler in v1.Source would have been difficult, as different scaling technologies support different options, and
  c) to minimize controversies and move the conversation forward (faster)...
  ...

• Note that we are not depending on KEDA anywhere here. If folks like this idea first, I will try to upstream some other helper functionality that will be useful for other folks interested in KEDA-based Sources (e.g., reconcile methods for scaledobjects and deployments, the listableTracker in eventing, a Resource duck type to track any k8s resource (in this case, scaledObjects), a discovery method to see if KEDA is installed, etc). All those are in knative-gcp, where I have a working prototype. See Add KEDA support for GCP Sources google/knative-gcp#551 if interested.

• Last but not least, I'd be happy to help @aslom and other folks with the KafkaSource as well. IMO it would be great to have this opt-in scaling support in our Pull-based Sources available rather soon-ish so that folks can start trying things out.

/cc @aslom @vaikas @n3wscott @lionelvillard @matzew @grantr

[knative-prow-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: nachocano
To complete the pull request process, please assign vaikas
You can assign the PR to them by writing /assign @vaikas in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[knative-metrics-robot]

The following is the coverage report on the affected files.
Say /test pull-knative-pkg-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
apis/duck/v1alpha1/scalable_source_types.go	Do not exist	0.0%
reconciler/filter.go	Do not exist	100.0%

[knative-prow-robot]

@nachocano: The following test failed, say /retest to rerun all failed tests:

Test name	Commit	Details	Rerun command
pull-knative-pkg-go-coverage	c9839f1	link	/test pull-knative-pkg-go-coverage

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[n3wscott]

why is the Keda?

Also, why Source?

It is just a Scalable ducktype. Separate the source from the Scaling parts.

[nachocano]

why is the Keda? --> Because it has keda-specific configuration

why Source --> because it implements the Source duck type.

why not just Scalable --> because is hard to do a generic Scalable one, different implementations have different configs.

[n3wscott]

I think Knative is the wrong repo to put this in.

Give this ducktype to KEDA?

[n3wscott]

/hold

This is not a ducktype. This is the KEDA api projected on an object. The fact that it is called KEDA is the hint that it is not generic enough to be a duck.

[nachocano]

I think Knative is the wrong repo to put this in.

Give this ducktype to KEDA?

/hold

This is not a ducktype. This is the KEDA api projected on an object. The fact that it is called KEDA is the hint that it is not generic enough to be a duck.

Not generic enough to be a KEDA duck from where you can implement KEDA-based sources?

[aslom]

@nachocano what do you think about using annotations for scaling configuration? That would keep it consistent with serving scaling API and not make it dependent on KEDA configuration naming? knative/eventing#2153 (comment)

@n3wscott Is it possible ot have duct type for annotations - validate in webhook and do mapping to annotation name/values with typing so MinScale is positive number for sutoscaling.knative.dev/minScale: "2"

[nachocano]

@nachocano what do you think about using annotations for scaling configuration? That would keep it consistent with serving scaling API and not make it dependent on KEDA configuration naming? knative/eventing#2153 (comment)

Thanks for the pointer @aslom. Let me check that out. Haven't seen what serving is doing...

Confirming the always polite holds from Sir Scotty...

[nachocano]

/close

I think folks are right, no need for this stuff. Annotations should suffice. Thanks @aslom for the pointer!
Created a separate PR for upstreaming the filter functions, and will update our knative-gcp to use annotations.