Channel conformance tests for spec.subscriber #3050

aliok · 2020-04-26T23:35:25Z

Proposed Changes

Conformance tests for channels' spec.subscriber (or spec.subscribable.subscriber on v1alpha1 channel duck type support
This test is not run for generic Channel see spec change here: https://github.com/knative/eventing/pull/3132/files

aliok · 2020-04-26T23:59:39Z

Meta Channel (channel.messaging.knative.dev/v1alpha1) doesn't allow specifying the subscribers in the spec, thus the conformance test fails.

Following is a simplified example of the failure:

cat <<EOS |kubectl apply -f -
---
apiVersion: messaging.knative.dev/v1alpha1
kind: Channel
metadata:
  name: foo
spec:
  subscribable:
    subscribers:
    - UID: "1234"
      ReplyUri: "foo.bar"
EOS

ends up in

Error from server (BadRequest): error when creating "channel_v1alpha1.yaml": admission webhook "validation.webhook.eventing.knative.dev" denied the request: validation failed: must not set the field(s): spec.subscribable.subscribers

Created #3051

matzew · 2020-04-27T12:44:10Z

test/conformance/helpers/channel_spec_test_helper.go

+		}
+
+		// SPEC: each channel CRD MUST contain an array of subscribers: spec.subscribers
+		channelable.Spec.Subscribers = []eventingduckv1beta1.SubscriberSpec{


@lionelvillard the spec.subscribers are only populated by the Subscription, right ?

Some new info: #3051 (comment)

aliok · 2020-05-05T22:28:06Z

/retest

matzew · 2020-05-08T02:55:21Z

/retest

aliok · 2020-05-08T06:40:31Z

#3051 is fixed

This PR is rebased now, let's see what happens

UPDATE: that fixed issue is not related to this one but it is related to #3037

lberk · 2020-05-11T19:35:46Z

once #3129 is merged, retest and I'm hoping this conformance test should pass (passed locally 🤞 )

lberk · 2020-05-11T20:06:04Z

/reopen
/retest

knative-prow-robot · 2020-05-11T20:06:10Z

@lberk: Reopened this PR.

In response to this:

/reopen
/retest

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

lberk · 2020-05-11T20:34:07Z

/test pull-knative-eventing-integration-tests

aliok · 2020-05-12T06:52:38Z

@lberk
This PR won't pass until we change the spec and also change the tests themselves to skip the CRD spec.subscribers tests for Channel. See #3051

#3129 is again about status.subscribers (I think), not spec.subscribers.

lberk · 2020-05-12T14:13:19Z

I chatted with @aliok this morning, and perhaps I need a bit more clarification as I look over this PR (and how it relates to #3051)

Relevant part of the spec is:

Spec Requirements
v1alpha1 Spec
Each channel CRD MUST contain an array of subscribers: spec.subscribable.subscribers

v1beta1 Spec
Each channel CRD MUST contain an array of subscribers: spec.subscribers

Part of this conformance test sets these subscriber fields directly (which is landing in an error[1]). I think in order to correct this conformance test, we need to have the client create a subscription targeting the channel, and then verify that the channel then mirrors that subscription to the spec array. Not set the subscribers field directly.

To me this means the spec is still valid (though perhaps could be tweaked to suggest the array of subscribers should not be set directly, but via subscriptions).

Thoughts? Suggestions? @aliok @matzew @vaikas

[1] - TestChannelSpec/TestChannelSpec-{Channel_messaging.knative.dev/v1alpha1}: channel_spec_test_helper.go:119: Error updating {"Channel" "messaging.knative.dev/v1alpha1"} with subscribers in spec: "admission webhook \"validation.webhook.eventing.knative.dev\" denied the request: validation failed: must not set the field(s): spec.subscribable.subscribers"

aliok · 2020-05-12T14:18:53Z

we need to have the client create a subscription targeting the channel, and then verify that the channel then mirrors that subscription to the spec array. Not set the subscribers field directly.

Yeah, I can change the test that way.

Create channel
Create subscription
Wait for the subscription to settle
Check the Channel CR spec.subscribers

aliok · 2020-05-14T08:48:45Z

/assign @lberk

want to review?

antoineco

Nice tests! A few nits.

test/conformance/helpers/channel.go

antoineco · 2020-05-14T09:01:53Z

test/conformance/helpers/channel_spec_test_helper.go

+	})
+}
+
+func channelSpecAllowsSubscribersArray(st *testing.T, client *lib.Client, channel metav1.TypeMeta, options ...lib.SetupClientOption) {


I'm curious about the reason behind this st variable name when other tests seem to unanimously use t.

single or specific test perhaps? I've seen tt vs t used in other spots as well iirc, I'm not sure we need to bike shed the variable name too much?

Like I said, just nits

I've seen that's setup that way in other conformance tests and used it that way.

But I think it could make sense.

st is defined here in the test runner call: https://github.com/knative/eventing/pull/3050/files#diff-5d560f68ffc10b7bccae5ab980f0ffddR41

Let's ignore that here in this PR and if we would like to change that var name, let's do that in a separate PR for all other tests as well.

antoineco · 2020-05-14T09:07:33Z

test/conformance/helpers/channel_spec_test_helper.go

+	client.WaitForResourceReadyOrFail(channelName, &channel)
+
+	sampleUrl, _ := apis.ParseURL("http://example.com")
+	gvr, _ := meta.UnsafeGuessKindToResource(channel.GroupVersionKind())


import "knative.dev/eventing/pkg/apis/messaging" // ... gvr := messaging.ChannelsResource.String()

channel here is actually passed by the test runner and it is different in each execution of the test.

IMC v1alpha1

IMC v1beta1

Channel v1alpha1

OK makes sense now

antoineco · 2020-05-14T09:08:19Z

test/conformance/helpers/channel_spec_test_helper.go

+		// treat missing annotation value as v1alpha1, as written in the spec
+		channelable, err := getChannelAsV1Alpha1Channelable(channelName, client, channel)
+		if err != nil {
+			st.Fatalf("Unable to get channel %q to v1alpha1 duck type: %q", channel, err)


The error doesn't need quotes I believe. (comment applies to other occurrences as well)

Converted all of them now to %s

antoineco · 2020-05-14T09:09:05Z

test/conformance/helpers/channel_spec_test_helper.go

+				{
+					UID:      "1234",
+					ReplyURI: sampleUrl,
+				},


Is it even necessary to add an element to the array for this assertion?

I think it is actually better to have an element inside the array instead of setting an empty array

antoineco · 2020-05-15T07:23:09Z

/lgtm

matzew · 2020-05-15T08:43:53Z

/lgtm
/approve

knative-prow-robot · 2020-05-15T08:44:05Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: aliok, matzew

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~test/OWNERS~~ [matzew]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

googlebot added the cla: yes Indicates the PR's author has signed the CLA. label Apr 26, 2020

knative-prow-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Apr 26, 2020

knative-prow-robot requested review from chaodaiG and chizhg April 26, 2020 23:35

knative-prow-robot added the area/test-and-release Test infrastructure, tests or release label Apr 26, 2020

aliok mentioned this pull request Apr 27, 2020

Meta Channel doesn't conform the spec's CRD spec.subscribers requirement #3051

Closed

matzew reviewed Apr 27, 2020

View reviewed changes

aliok mentioned this pull request May 5, 2020

Conformance tests for channel CRD spec requirements #2988

Closed

1 task

aliok force-pushed the channel_conformance_test_spec_subscriber branch from 123b020 to ad7673e Compare May 8, 2020 06:39

lberk mentioned this pull request May 11, 2020

Fix Channel assuming v1alpha1 backing channel if no duck annotation #3129

Merged

knative-prow-robot closed this in #3129 May 11, 2020

knative-prow-robot reopened this May 11, 2020

lberk mentioned this pull request May 12, 2020

Add note to channel spec regarding channel subscription spec #3132

Merged

knative-prow-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 14, 2020

aliok added 2 commits May 14, 2020 10:15

Channel conformance tests for spec.subscriber

fb5518b

Skip channel conformance tests for spec.subscriber for generic Channel

28b90a9

aliok force-pushed the channel_conformance_test_spec_subscriber branch from 0041836 to 28b90a9 Compare May 14, 2020 07:17

knative-prow-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 14, 2020

knative-prow-robot assigned lberk May 14, 2020

antoineco reviewed May 14, 2020

View reviewed changes

aliok added 2 commits May 15, 2020 09:54

Get rid of hardcoded Channel GVK strings

c1f2ea6

Change %q with %s

87ccd27

knative-prow-robot assigned antoineco May 15, 2020

knative-prow-robot added the lgtm Indicates that a PR is ready to be merged. label May 15, 2020

knative-prow-robot assigned matzew May 15, 2020

knative-prow-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 15, 2020

knative-prow-robot merged commit f624e65 into knative:master May 15, 2020

aliok deleted the channel_conformance_test_spec_subscriber branch May 15, 2020 11:56

aliok mentioned this pull request May 15, 2020

Channel conformance test spec subscriber knative/eventing-contrib#1232

Merged

lberk mentioned this pull request Sep 1, 2020

Add lberk to approvers #3165

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Channel conformance tests for spec.subscriber #3050

Channel conformance tests for spec.subscriber #3050

aliok commented Apr 26, 2020 •

edited

Loading

aliok commented Apr 26, 2020 •

edited

Loading

matzew Apr 27, 2020

aliok Apr 30, 2020

aliok commented May 5, 2020

matzew commented May 8, 2020

aliok commented May 8, 2020 •

edited

Loading

lberk commented May 11, 2020

lberk commented May 11, 2020

knative-prow-robot commented May 11, 2020

lberk commented May 11, 2020

aliok commented May 12, 2020 •

edited

Loading

lberk commented May 12, 2020

aliok commented May 12, 2020

aliok commented May 14, 2020

antoineco left a comment

antoineco May 14, 2020

lberk May 14, 2020

antoineco May 14, 2020

aliok May 15, 2020

antoineco May 14, 2020 •

edited

Loading

aliok May 15, 2020

antoineco May 15, 2020

antoineco May 14, 2020

aliok May 15, 2020

antoineco May 14, 2020

aliok May 15, 2020

antoineco commented May 15, 2020

matzew commented May 15, 2020

knative-prow-robot commented May 15, 2020

Channel conformance tests for spec.subscriber #3050

Channel conformance tests for spec.subscriber #3050

Conversation

aliok commented Apr 26, 2020 • edited Loading

Proposed Changes

aliok commented Apr 26, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aliok commented May 5, 2020

matzew commented May 8, 2020

aliok commented May 8, 2020 • edited Loading

lberk commented May 11, 2020

lberk commented May 11, 2020

knative-prow-robot commented May 11, 2020

lberk commented May 11, 2020

aliok commented May 12, 2020 • edited Loading

lberk commented May 12, 2020

aliok commented May 12, 2020

aliok commented May 14, 2020

antoineco left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

antoineco May 14, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

antoineco commented May 15, 2020

matzew commented May 15, 2020

knative-prow-robot commented May 15, 2020

aliok commented Apr 26, 2020 •

edited

Loading

aliok commented Apr 26, 2020 •

edited

Loading

aliok commented May 8, 2020 •

edited

Loading

aliok commented May 12, 2020 •

edited

Loading

antoineco May 14, 2020 •

edited

Loading