PhysicalChannelSyncFailed when mixing v1alpha1 and v1beta1 Subscriptions

[matzew]

Describe the bug

Getting PhysicalChannelSyncFailed error, when mixing alpha/beta APIs on subscribers, see:

---
apiVersion: messaging.knative.dev/v1beta1
kind: InMemoryChannel
metadata:
  name: messagesv1beta1
---
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: event-display
spec:
  template:
    spec:
      containers:
      - image: quay.io/openshift-knative/knative-eventing-sources-event-display:v0.13.2
---
apiVersion: messaging.knative.dev/v1alpha1
kind: Subscription
metadata:
  name: a-b-as-a
spec:
  channel:
    apiVersion: messaging.knative.dev/v1alpha1
    kind: InMemoryChannel
    name: messagesv1beta1
  subscriber:
    ref:
      apiVersion: serving.knative.dev/v1
      kind: Service
      name: event-display

Expected behavior

Not sure, tbh ... since IMO the usage of the channel is wrong:

...
    apiVersion: messaging.knative.dev/v1alpha1
    kind: InMemoryChannel
    name: messagesv1beta1

Or should this work and be "converted" ?

/cc @vaikas
/cc @lionelvillard

Knative release version

• 0.13
• 0.14

[nicolaferraro]

This happens in Camel K as well. The inmemorychannel is both served and stored in version v1alpha1, so I'd expect that I can reference it that way.

[matzew]

and, even if you query via version, you get the channel, see:

k get inmemorychannel.v1alpha1.messaging.knative.dev        

NAME              READY   REASON   URL                                                           AGE
messagesv1beta1   True             http://messagesv1beta1-kn-channel.default.svc.cluster.local   60s

and via beta, same object:

k get inmemorychannel.v1beta1.messaging.knative.dev

NAME              READY   REASON   URL                                                           AGE
messagesv1beta1   True             http://messagesv1beta1-kn-channel.default.svc.cluster.local   66s

[vaikas]

The channels should have the label indicating if they do v1alpha1 or v1beta1. I think we do this in the IMC conversions.

[sebgoa]

Yeah...I cannot create triggers anymore :(

Failed to sync physical Channel: admission webhook "webhook.eventing.knative.dev" denied the request: mutation failed: cannot decode incoming new object: json: unknown field "subscribers"

Which I get with reason PhysicalChannelSyncFailed

[vaikas]

Ok, I have a fix for this. I'll try to get all the tests, etc. done today, but if not, Monday.

[jchesterpivotal]

I just hit this too. It appeared after restarting a cluster, but I don't know if that's related or coincidental.

[vaikas]

Can @sebgoa & @jchesterpivotal send me a repro, esp. if it's related to triggers as I wouldn't expect that to fail. So, I want to make sure I cover those in my tests. The @matzew example I can totes understand.

[jchesterpivotal]

$ kn service create cloudevents-player --image ruromero/cloudevents-player:latest --env BROKER_URL=http://default-broker

# snip

$ kn trigger create cloudevents-player --sink cloudevents-player

# snip

$ kn trigger describe cloudevents-player --log-http                                                                                                                  
===== REQUEST =====
GET /apis/eventing.knative.dev/v1alpha1/namespaces/default/triggers/cloudevents-player HTTP/1.1
Host: [redacted]
User-Agent: kn/v0.0.0 (darwin/amd64) kubernetes/$Format
Accept: application/json, */*
Authorization: ********
Accept-Encoding: gzip


===== RESPONSE =====
HTTP/2.0 200 OK
Content-Length: 1729
Audit-Id: 6d2f3d57-6937-4d1d-850c-9cbde64d061a
Content-Type: application/json
Date: Fri, 24 Apr 2020 20:59:22 GMT

{"apiVersion":"eventing.knative.dev/v1alpha1","kind":"Trigger","metadata":{"annotations":{"eventing.knative.dev/creator":"[redacted]","eventing.knative.dev/lastModifier":"[redacted]"},"creationTimestamp":"2020-04-24T20:36:45Z","generation":1,"labels":{"eventing.knative.dev/broker":"default"},"name":"cloudevents-player","namespace":"default","resourceVersion":"4378171","selfLink":"/apis/eventing.knative.dev/v1alpha1/namespaces/default/triggers/cloudevents-player","uid":"cec486c5-c6e6-4942-8a3b-f61f4503a356"},"spec":{"broker":"default","filter":{},"subscriber":{"ref":{"apiVersion":"serving.knative.dev/v1","kind":"Service","name":"cloudevents-player","namespace":"default"}}},"status":{"conditions":[{"lastTransitionTime":"2020-04-24T20:36:45Z","status":"True","type":"BrokerReady"},{"lastTransitionTime":"2020-04-24T20:36:45Z","status":"True","type":"DependencyReady"},{"lastTransitionTime":"2020-04-24T20:36:45Z","message":"Failed to sync physical Channel: admission webhook \"webhook.eventing.knative.dev\" denied the request: mutation failed: cannot decode incoming new object: json: unknown field \"subscribers\"","reason":"PhysicalChannelSyncFailed","status":"False","type":"Ready"},{"lastTransitionTime":"2020-04-24T20:36:45Z","status":"True","type":"SubscriberResolved"},{"lastTransitionTime":"2020-04-24T20:36:45Z","message":"Failed to sync physical Channel: admission webhook \"webhook.eventing.knative.dev\" denied the request: mutation failed: cannot decode incoming new object: json: unknown field \"subscribers\"","reason":"PhysicalChannelSyncFailed","status":"False","type":"SubscriptionReady"}],"observedGeneration":1,"subscriberURI":"http://cloudevents-player.default.svc.cluster.local"}}

 * * * * * *
Name:         cloudevents-player
Namespace:    default
Labels:       eventing.knative.dev/broker=default
Annotations:  eventing.knative.dev/creator=[redacted], eventing.knative.dev/lastModifier ...
Age:          22m
Broker:       default

Sink:
  Name:       cloudevents-player
  Namespace:  default
  Resource:   Service (serving.knative.dev/v1)

Conditions:
  OK TYPE                  AGE REASON
  !! Ready                 22m PhysicalChannelSyncFailed
  ++ BrokerReady           22m
  ++ DependencyReady       22m
  ++ SubscriberResolved    22m
  !! SubscriptionReady     22m PhysicalChannelSyncFailed

[vaikas]

vaikas-a01:3014-repro vaikas$ kubectl -n vaikas apply -f ./repro.yaml
inmemorychannel.messaging.knative.dev/messagesv1beta1 created
service.serving.knative.dev/event-display created
subscription.messaging.knative.dev/a-b-as-a created
vaikas-a01:3014-repro vaikas$ kubectl -n vaikas get subscriptions
NAME       READY   REASON   AGE
a-b-as-a   True             8s
vaikas-a01:3014-repro vaikas$ kubectl -n vaikas get inmemorychannels -oyaml
apiVersion: v1
items:
- apiVersion: messaging.knative.dev/v1beta1
  kind: InMemoryChannel
  metadata:
    annotations:
      kubectl.kubernetes.io/last-applied-configuration: |
        {"apiVersion":"messaging.knative.dev/v1beta1","kind":"InMemoryChannel","metadata":{"annotations":{},"name":"messagesv1beta1","namespace":"vaikas"}}
      messaging.knative.dev/creator: vaikas@vmware.com
      messaging.knative.dev/lastModifier: system:serviceaccount:knative-eventing:eventing-controller
      messaging.knative.dev/subscribable: v1alpha1
    creationTimestamp: "2020-04-25T20:29:10Z"
    generation: 2
    name: messagesv1beta1
    namespace: vaikas
    resourceVersion: "523570"
    selfLink: /apis/messaging.knative.dev/v1beta1/namespaces/vaikas/inmemorychannels/messagesv1beta1
    uid: 6b2776f7-b294-465e-966c-233bce0e0014
  spec:
    subscribers:
    - delivery:
        deadLetterSink: {}
      generation: 1
      subscriberUri: http://event-display.vaikas.svc.cluster.local
      uid: 78b95e53-8bcc-494c-9c8e-fb9e16d7a5b8
  status:
    address:
      url: http://messagesv1beta1-kn-channel.vaikas.svc.cluster.local
    conditions:
    - lastTransitionTime: "2020-04-25T20:29:10Z"
      status: "True"
      type: Ready
    observedGeneration: 2
    subscribers:
    - observedGeneration: 1
      ready: "True"
      uid: 78b95e53-8bcc-494c-9c8e-fb9e16d7a5b8
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""
vaikas-a01:3014-repro vaikas$ kubectl -n vaikas get subscriptions -oyaml
apiVersion: v1
items:
- apiVersion: messaging.knative.dev/v1beta1
  kind: Subscription
  metadata:
    annotations:
      kubectl.kubernetes.io/last-applied-configuration: |
        {"apiVersion":"messaging.knative.dev/v1alpha1","kind":"Subscription","metadata":{"annotations":{},"name":"a-b-as-a","namespace":"vaikas"},"spec":{"channel":{"apiVersion":"messaging.knative.dev/v1alpha1","kind":"InMemoryChannel","name":"messagesv1beta1"},"subscriber":{"ref":{"apiVersion":"serving.knative.dev/v1","kind":"Service","name":"event-display"}}}}
      messaging.knative.dev/creator: vaikas@vmware.com
      messaging.knative.dev/lastModifier: vaikas@vmware.com
    creationTimestamp: "2020-04-25T20:29:10Z"
    finalizers:
    - subscriptions.messaging.knative.dev
    generation: 1
    name: a-b-as-a
    namespace: vaikas
    resourceVersion: "523575"
    selfLink: /apis/messaging.knative.dev/v1beta1/namespaces/vaikas/subscriptions/a-b-as-a
    uid: 78b95e53-8bcc-494c-9c8e-fb9e16d7a5b8
  spec:
    channel:
      apiVersion: messaging.knative.dev/v1alpha1
      kind: InMemoryChannel
      name: messagesv1beta1
    subscriber:
      ref:
        apiVersion: serving.knative.dev/v1
        kind: Service
        name: event-display
        namespace: vaikas
  status:
    conditions:
    - lastTransitionTime: "2020-04-25T20:29:14Z"
      status: "True"
      type: AddedToChannel
    - lastTransitionTime: "2020-04-25T20:29:14Z"
      status: "True"
      type: ChannelReady
    - lastTransitionTime: "2020-04-25T20:29:14Z"
      status: "True"
      type: Ready
    - lastTransitionTime: "2020-04-25T20:29:14Z"
      status: "True"
      type: ReferencesResolved
    observedGeneration: 1
    physicalSubscription:
      subscriberURI: http://event-display.vaikas.svc.cluster.local
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""

[vaikas]

@nicolaferraro when you say this happens with camel-k, I only see a source in the eventing-contrib, do you have a repro that I could try to look at, or did this fix your issue as well?