Question: Could I use trigger's filter, to filter from "data" field?

[usrdmn]

Hello!

I have a Kafka broker aggregating all my messages from different Kafka instances and topics using kafkasources. My main goal is to deliver these messages to appropriate functions (sequences, parallels, etc.) using triggers.

Initially, I thought it was only possible by filtering on cloud-native attributes, but then I found this document that seems to enable message filtering based on data fields.

Here's an example from the document:

spec:
  filter:
    expression: >
      user.id == "abc123"

This is similar to a YAML file I created for testing, which is supposed to trigger the test-function function:

apiVersion: eventing.knative.dev/v1
kind: Trigger
metadata:
  name: test-trigger-with-filter
  namespace: knative
spec:
  broker: kafka-broker-external
  filter:
    expression: >
      user.id == "abc123"
  subscriber:
    ref:
      apiVersion: serving.knative.dev/v1
      kind: Service
      name: test-function

However, when I try to apply this YAML file to my cluster, I get an error:

Error from server (BadRequest): error when creating "trigger-with-filter.yaml": Trigger in version "v1" cannot be handled as a Trigger: strict decoding error: unknown field "spec.filter.expression"

I have an almost default installation of Serving and Eventing.

To summarize, my question is: Do Knative Eventing triggers support filtering messages based on specific values in the data field?

[pierDipi]

Hi @usrdmn, filtering on the data field is not possible currently but we would be open to a proposal for it, it seems this would be related to #8001 or #7704

[Cali0707]

@pierDipi at least for the CESQL side of stuff, there was a lot of opposition to introducing the ability to filter on data when I brought it up in the serverless wg call a while ago. Mostly re: not tying to CESQL spec to a specific data format

[pierDipi]

@Cali0707 we could explore to see if we can extend it with custom functions?

[Cali0707]

@Cali0707 we could explore to see if we can extend it with custom functions?

Sure, do you want to discuss in the WG call tomorrow @pierDipi ?