diff --git a/specs/eventing/data-plane.md b/specs/eventing/data-plane.md index 81e1aa3fc..f825983a5 100644 --- a/specs/eventing/data-plane.md +++ b/specs/eventing/data-plane.md @@ -1,125 +1,182 @@ -# Knative Eventing Data Plane Contracts +# Knative Eventing Data Plane Contract -## Introduction - -Developers using Knative Eventing need to know what is supported for delivery to -user provided components that receive events. Knative Eventing defines contract -for data plane components and we have listed them here. - -## Conformance - -The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", -"SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be -interpreted as described in RFC2119. - -## Data plane contract for Sinks +## Terminology -A **Sink** MUST be able to handle duplicate events. +This document discusses communication between two parties: -A **Sink** is an [_addressable_](./interfaces.md#addressable) resource that -takes responsibility for the event. A Sink could be a consumer of events, or -middleware. A Sink MUST be able to receive CloudEvents over HTTP and HTTPS. +- **Event Senders** initiate an HTTP POST to deliver a CloudEvent. +- **Event Recipients** receive an HTTP POST and accept (or reject) a CloudEvent. -A **Sink** MAY be [_callable_](./interfaces.md#callable) resource that -represents an Addressable endpoint which receives an event as input and -optionally returns an event to forward downstream. +Additionally, these roles can be combined in different ways: -Almost every component in Knative Eventing may be a Sink providing -composability. +- **Event Processors** can be event senders, event recipients, or both. +- **Event Sources** are exclusively event senders, and never act as recipients. +- **Event Sinks** are exclusively event recipients, and never act as senders. -Every Sink MUST support HTTP Protocol Binding for CloudEvents -[version 1.0](https://github.com/cloudevents/spec/blob/v1.0/http-protocol-binding.md) -and -[version 0.3](https://github.com/cloudevents/spec/blob/v0.3/http-transport-binding.md) -with restrictions and extensions specified below. - -### HTTP Support - -This section adds restrictions on -[requirements in HTTP Protocol Binding for CloudEvents](https://github.com/cloudevents/spec/blob/v1.0/http-protocol-binding.md#12-relation-to-http). +## Introduction -Sinks MUST accept HTTP requests with POST method and MAY support other HTTP -methods. If a method is not supported Sink MUST respond with HTTP status code -`405 Method Not Supported`. Non-event requests (e.g. health checks) are not -constrained. +Late-binding event senders and recipients (composing applications using +configuration) only works when all event senders and recipients speak a common +protocol. In order to enable wide support for senders and recipients, Knative +Eventing extends the +[CloudEvents HTTP bindings](https://github.com/cloudevents/spec/blob/v1.0.1/http-protocol-binding.md) +with additional semantics for the following reasons: -The URL used by a Sink MUST correspond to a single, unique endpoint at any given -moment in time. This MAY be done via the host, path, query string, or any -combination of these. This mapping is handled exclusively by the -[Addressable control-plane](./interfaces.md#control-plane) exposed via the -`status.address.url`. +- Knative Eventing aims to enable at least once event processing; hence it + prefers duplicate delivery to discarded events. The CloudEvents spec does not + take a stance here. -If an HTTP request's URL does not correspond to an existing endpoint, then the -Sink MUST respond with `404 Not Found`. +- The CloudEvents HTTP bindings provide a relatively simple and efficient + network protocol which can easily be supported in a wide variety of + programming languages leveraging existing library investments in HTTP. The + CloudEvents project has already written these libraries for many popular + languages. -Every non-Callable Sink MUST respond with `202 Accepted` if the request is -accepted. +- Knative Eventing assumes a sender-driven (push) event delivery system. That + is, each recipient is actively responsible for an event until it is handled + (or affirmatively delivered to all following recipients). -If Sink is Callable it MAY respond with `200 OK` and a single event in the HTTP -response. A returned event is not required to be related to the received event. -The Callable should return a successful response if the event was processed -successfully. If there is no event to send back then Callable Sink MUST respond -with 2xx HTTP and with empty body. +- Knative Eventing aims to make [event sources](./overview.md#event-source) and + event-processing software easier to write; as such, it imposes higher + standards on system components like [brokers](./overview.md#broker) and + [channels](./overview.md#channel) than on edge components. -If a Sink receives a request and is unable to parse a valid CloudEvent, then it -MUST respond with `400 Bad Request`. +This contract defines a mechanism for a single event sender to reliably deliver +a single event to a single recipient. Building from this primitive, chains of +reliable event delivery and event-driven applications can be built. -### Content Modes Supported +## Background -A Sink MUST support `Binary Content Mode` and `Structured Content Mode` as -described in -[HTTP Message Mapping section of HTTP Protocol Binding for CloudEvents](https://github.com/cloudevents/spec/blob/master/http-protocol-binding.md#3-http-message-mapping) +The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", +"SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be +interpreted as described in +[RFC2119](https://datatracker.ietf.org/doc/html/rfc2119). -A Sink MAY support `Batched Content Mode` but that mode is not used in Knative -Eventing currently (that may change in future). +When not specified in this document, the +[CloudEvents HTTP bindings, version 1.0](https://github.com/cloudevents/spec/blob/v1.0.1/http-protocol-binding.md) +and [HTTP 1.1 protocol](https://tools.ietf.org/html/rfc7230) standards MUST be +followed (with the CloudEvents bindings taking precedence in the case of +conflict). -### Retries +The current version of this document does not describe protocol negotiation or +any delivery mechanism other than HTTP 1.1. Future versions might define +protocol negotiation to optimize delivery; compliant implementations SHOULD aim +to interoperate by ignoring unrecognized negotiation options (such as +[HTTP `Upgrade` headers](https://datatracker.ietf.org/doc/html/rfc7230#section-6.7)). -Sinks should expect that retries and accept possibility that duplicate events -may be delivered. +## Event Delivery -### Error handling +### Minimum supported protocol -If Sink is not returning HTTP success header (200 or 202) then the event may be -sent again. If the event can not be delivered then some sources of events (such -as Knative sources, brokers or channels) MAY support -[dead letter sink or channel](https://github.com/knative/eventing/blob/main/docs/delivery/README.md) for events that can not be -delivered. +All senders and recipients MUST support the CloudEvents 1.0 protocol and the +[binary](https://github.com/cloudevents/spec/blob/v1.0.1/http-protocol-binding.md#31-binary-content-mode) +and +[structured](https://github.com/cloudevents/spec/blob/v1.0.1/http-protocol-binding.md#32-structured-content-mode) +content modes of the CloudEvents HTTP binding. Senders which do not advertise +the ability to accept [reply events](#derived-reply-events) MAY implement only +one content mode, as the recipient is not allowed to negotiate the content mode. + +### HTTP Verbs + +In the absence of specific delivery preferences, the sender MUST initiate +delivery of the event to the recipient using the HTTP POST verb, using either +the structured or binary encoding of the event (sender's choice). This delivery +MUST be performed using the CloudEvents HTTP Binding, version 1.x. + +Senders MAY probe the recipient with an +[HTTP OPTIONS request](https://tools.ietf.org/html/rfc7231#section-4.3.7); if +implemented, the recipient MUST indicate support for the POST verb using the +[`Allow` header](https://tools.ietf.org/html/rfc7231#section-7.4.1). Senders +which receive an error when probing with HTTP OPTIONS SHOULD proceed using the +HTTP POST mechanism. + +### Event Acknowledgement and Delivery Retry + +Event recipients MUST use the HTTP response code to indicate acceptance of an +event. The recipient SHOULD NOT return a response accepting the event until it +has handled the event (processed the event or stored it in stable storage). The +following response codes are explicitly defined; event recipients MAY also +respond with other response codes. A response code not in this table SHOULD be +treated as a retriable error. + +| Response code | Meaning | Retry | Delivery completed | Error | +| ------------- | ------------------------------------------------- | ----- | ------------------ | ----- | +| `1xx` | (Unspecified) | No\* | No\* | Yes\* | +| `200` | [Accepted, event in reply](#derived-reply-events) | No | Yes | No | +| `202` | Event accepted | No | Yes | No | +| other `2xx` | (Unspecified) | No | Yes | No | +| `3xx` | (Unspecified) | No\* | No\* | Yes\* | +| `400` | Unparsable event | No | No | Yes | +| `404` | Endpoint does not exist | Yes | No | Yes | +| `409` | Conflict / Processing in progress | Yes | No | Yes | +| `429` | Too Many Requests / Overloaded | Yes | No | Yes | +| other `4xx` | Error | No | No | Yes | +| `5xx` | Error | Yes | No | Yes | + +\* Unspecified `1xx`, `2xx`, and `3xx` response codes are **reserved for future +extension**. Event recipients SHOULD NOT send these response codes in this spec +version, but event senders MUST handle these response codes as errors or success +as appropriate and implement described success or failure behavior. + +Recipients MUST accept duplicate delivery of events, but they are not REQUIRED +to detect that they are duplicates. If duplicate detection is implemented, then +as specified in the +[CloudEvents specification](https://github.com/cloudevents/spec/blob/v1.0.1/primer.md#id), +event recipients MUST use the +[`source` and `id` attributes](https://github.com/cloudevents/spec/blob/v1.0.1/spec.md#required-attributes) +to identify duplicate events. This specification does not describe state +requirements for recipients which need to detect duplicate events. In general, +senders MAY add or update other CloudEvent attributes on each delivery attempt; +see [observability](#observability) for an example case. + +Where possible, event senders SHOULD re-attempt delivery of events where the +HTTP request returned a retryable status code. It is RECOMMENDED that event +senders implement some form of congestion control (such as exponential backoff) +and delivery throttling when managing retry timing. Congestion control MAY cause +event delivery to fail or MAY include not retrying failed delivery attempts. +This specification does not document any specific congestion control algorithm +or parameters. [Brokers](./overview.md#broker) and +[Channels](./overview.md#channel) MUST implement congestion control and MUST +implement retries. ### Observability -CloudEvents received by Sink MAY have -[Distributed Tracing Extension Attribute](https://github.com/cloudevents/spec/blob/v1.0/extensions/distributed-tracing.md). - -### Event reply contract - -An event sender supporting event replies SHOULD include a `Prefer: reply` header -in delivery requests to indicate to the sink that event reply is supported. An -event sender MAY ignore an event reply in the delivery response if the -`Prefer: reply` header was not included in the delivery request. - -An example is that a Broker supporting event reply sends events with an -additional header `Prefer: reply` so that the sink connected to the Broker knows -event replies will be accepted. While a source sends events without the header, -in which case the sink may assume that any event reply will be dropped without -error or retry attempt. If a sink wishes to ensure the reply events will be -delivered, it can check for the existence of the `Prefer: reply` header in the -delivery request and respond with an error code if the header is not present. - -### Data plane contract for Sources - -See [Source Delivery specification](sources.md#source-event-delivery) -for details. - -### Data plane contract for Channels - -See [Channel Delivery specification](channel.md#data-plane) for details. - -### Data plane contract for Brokers - -See [Broker Delivery specification](broker.md) - -## Changelog - -- 2020-04-20: `0.13.x release`: initial version that documents common contract - for sinks, sources, channels and brokers. +Event senders MAY add or update CloudEvents attributes before sending to +implement observability features such as tracing; in particular, the +`traceparent` and `tracestate` distributed tracing attributes defined by +[W3C](https://www.w3.org/TR/trace-context/) and +[CloudEvents](https://github.com/cloudevents/spec/blob/v1.0/extensions/distributed-tracing.md) +MAY be modified in this way for each delivery attempt of the same event. + +This specification does not mandate any particular logging or metrics +aggregation, nor a method of exposing observability information to users +configuring the resources. Platform administrators SHOULD expose event-delivery +telemetry to users through platform-specific interfaces, but such interfaces are +beyond the scope of this document. + +### Derived (Reply) Events + +In some applications, an event recipient MAY emit an event in reaction to a +received event. Senders MAY choose to support this pattern by accepting an +encoded CloudEvent in the HTTP response. + +An event sender MAY document support for this pattern by including a +`Prefer: reply` header in the HTTP POST request. This header indicates to the +event recipient that the caller will accept a +[`200` response](#event-acknowledgement-and-repeat-delivery) which includes a +CloudEvent encoded using the binary or structured formats. +[Brokers](./overview.md#broker) and [Channels](./overview.md#channel) MUST +indicate support for replies using the `Prefer: reply` header when sending to +the `spec.subscriber` address. + +A recipient MAY reply to any HTTP POST with a `200` response to indicate that +the event was processed successfully, with or without a response payload. If the +recipient will _never_ provide a response payload, the `202` response code is +also acceptable. Responses with a `202` response code MUST NOT be processed as +reply events. + +If a recipient chooses to reply to a sender with a `200` response code and a +reply event in the absence of a `Prefer: reply` header from the sender, the +sender SHOULD treat the event as accepted, and MAY log an error about the +unexpected payload. If a sender will process a reply event it MUST include the +`Prefer: reply` header on the POST request. diff --git a/specs/eventing/images/eventing-overview.svg b/specs/eventing/images/eventing-overview.svg new file mode 100644 index 000000000..4ba72413f --- /dev/null +++ b/specs/eventing/images/eventing-overview.svg @@ -0,0 +1,87 @@ +eventing.knative.devmessaging.knative.devBrokerTriggerTriggerChannelSubscriptionSubscription«Knative Service»Addressable«Kubernetes Service»Addressablebrokerbrokersubscriberchannelchannelsubscriberreply \ No newline at end of file diff --git a/specs/eventing/motivation.md b/specs/eventing/motivation.md index 9d2b48653..56c96e66a 100644 --- a/specs/eventing/motivation.md +++ b/specs/eventing/motivation.md @@ -1,49 +1,33 @@ # Motivation -The goal of Knative Eventing is to define common, composable primitives to -enable late-binding event sources and event consumers. +The goal of the Knative Eventing project is to define common primitives to +enable composing event-processing applications through configuration, rather +than application code. - +Building by combining independent components provides a number of benefits for +application designers: -Knative Eventing has following principles: - -1. Services are loosely coupled during development and deployed independently on - a variety of platforms (Kubernetes, VMs, SaaS or FaaS). +1. Services are loosely coupled during development and may be deployed + independently on a variety of platforms (Kubernetes, VMs, SaaS or FaaS). This + composability allows re-use of common patterns and building blocks, even + across programming language and tooling boundaries. 1. A producer can generate events before a consumer is listening, and a consumer can express an interest in an event or class of events that is not yet being - produced. + produced. This allows event-driven applications to evolve over time without + needing to closely coordinate changes. -1. Services can be connected to create new applications - - without modifying producer or consumer. +1. Services can be connected to create new applications: + - without modifying producer or consumer - with the ability to select a specific subset of events from a particular producer -These primitives enable producing and consuming events adhering to the -[CloudEvents Specification](https://github.com/cloudevents/spec), in a decoupled -way. - -Kubernetes has no primitives related to event processing, yet this is an -essential component in serverless workloads. Eventing introduces high-level -primitives for event production and delivery with an initial focus on push over -HTTP. If a new event source or type is required of your application, the effort -required to plumb them into the existing eventing framework will be minimal and -will integrate with CloudEvents middleware and message consumers. - -Knative eventing implements common components of an event delivery ecosystem: -enumeration and discovery of event sources, configuration and management of -event transport, and declarative binding of events (generated either by storage -services or earlier computation) to further event processing and persistence. - -The Knative Eventing API is intended to operate independently, and interoperate -well with the [Serving API](https://github.com/knative/serving) and -[Build API](https://github.com/knative/build). - ---- - -_Navigation_: +In order to enable loose coupling and late-binding of event producers and +consumers, Knative Eventing utilizes and extends the +[CloudEvents specification](https://github.com/cloudevents/spec) as the data +plane protocol between components. Knative Eventing prioritizes at-least-once +delivery semantics, using the CloudEvents HTTP POST (push) transport as a +minimum common transport between components. -- **Motivation and goals** -- [Resource type overview](overview.md) -- [Interface contracts](interfaces.md) -- [Object model specification](spec.md) +Knative Eventing also defines patterns to simplify the construction and usage of +event producers and consumers. diff --git a/specs/eventing/overview.md b/specs/eventing/overview.md index 3fafca4d4..87aa3836a 100644 --- a/specs/eventing/overview.md +++ b/specs/eventing/overview.md @@ -1,97 +1,152 @@ # Resource Types -The API defines and provides a complete implementation for -[Trigger](spec.md#kind-trigger), [Broker](spec.md#kind-broker), -[Subscription](spec.md#kind-subscription) and abstract resource definitions for -[Channels](spec.md#kind-channel). - -With extensibility and composability as a goal of Knative Eventing, the eventing -API defines several resources that can be reduced down to well understood -contracts. These eventing resource interfaces may be fulfilled by other -Kubernetes objects and then composed in the same way as the concrete objects. -The interfaces are ([Addressable](interfaces.md#addressable), -[Callable](interfaces.md#callable)). For more details, see -[Interface Contracts](interfaces.md). - -- A **Trigger** describes a filter on event attributes which should be delivered - to an _Addressable_. - -- A **Broker** provides a bucket of events which can be selected by attribute. - - - -![Broker Trigger Overview](images/broker-trigger-overview.svg) - -The above diagram shows a _Broker_ ingesting events and delivering to a -_Service_ only when the _Trigger_ filter matches. - -- A **Subscription** describes the transformation of an event and optional - forwarding of a returned event. - -- A **Channel** provides event persistence and fanout of events from a - well-known input address to multiple outputs described by _Subscriptions_. - - - -![Resource Types Overview](images/resource-types-overview.svg) - -Channels as well as Sources are defined by independent CRDs that can be -installed into a cluster. Both Sources and Channels implementations can be -created directly. A Channel however offers also a way to create the backing -implementation as part of the generic Channel (using a _channelTemplate_). - -## Trigger - -**Trigger** describes a registration of interest on a filter set of events -delivered to a _Broker_ which should be delivered to an _Addressable_. Events -selected by a _Trigger_ are buffered independently from other _Triggers_, even -if they deliver to the same _Addressable_. - -For more details, see [Kind: Trigger](spec.md#kind-trigger). - -## Broker - -**Broker** provides an eventing mesh. This allows producers to deliver events to -a single endpoint and not need to worry about the routing details for individual -consumers. - -For more details, see [Kind: Broker](spec.md#kind-broker). - -## Subscription - -**Subscriptions** describe a flow of events from one _Channel_ to the next -Channel\* through transformations (such as a Knative Service which processes -CloudEvents over HTTP). A _Subscription_ controller resolves the addresses of -transformations (`subscriber`) and destination storage (`reply`) through the -_Callable_ and _Addressable_ interface contracts, and writes the resolved -addresses to the _Channel_ in the `channel` reference. _Subscriptions_ do not -need to specify both a transformation and a storage destination, but at least -one must be provided. - -All event delivery linkage from a **Subscription** is 1:1 – only a single -`channel`, `subscriber`, and `reply` may be provided. - -For more details, see [Kind: Subscription](spec.md#kind-subscription). - -## Channel - -**Channel** provides an event delivery mechanism which can fan out received -events to multiple destinations via _Subscriptions_. A _Channel_ has a single -inbound _Addressable_ interface which may accept events delivered directly or -forwarded from multiple _Subscriptions_. Different _Channels_ may implement -different degrees of persistence. Event delivery order is dependent on the -backing implementation of the _Channel_. - -Event selection on a _Channel_ is 1:N – a single _Channel_ may fan out to -multiple _Subscriptions_. - -See [Kind: Channel](spec.md#kind-channel). - ---- - -_Navigation_: - -- [Motivation and goals](motivation.md) -- **Resource type overview** -- [Interface contracts](interfaces.md) -- [Object model specification](spec.md) +The Knative Eventing API provides primitives for two common event-processing +patterns: + +- Topology-based event routing ([`messaging.knative.dev`](#messaging)) + + Events are routed based on _connections between objects_ (in particular, + events flow along a [channel](#channel) to all + [subscriptions](#subscription)). This model can be thought of as "event + plumbing" in that events are managed like flows of water through pipes. In + order to enable event processing, subscriptions provide a mechanism to route + "reply" events further through the object topology. + +- Content-based event routing ([`eventing.knative.dev`](#eventing)) + + Events are selected for routing based on the event _attributes_ rather than + primarily by object connections (a [broker](#broker) provides a stream of + events which can be selected by a [trigger](#trigger)). This model is more + akin to picking parts off a conveyor belt, where each event is considered + separately for processing. Content-based routing handles reply events by + re-enqueuing the events in the originating Broker. + +Knative Eventing does not directly specify mechanisms for other event-processing +models, including multi-stage workflows, correlated request-reply, and +sequential (windowed) event processing; these models could be built using the +primitives provided by Knative, or Knative could deliver events to an external +system that implements these models. + +In addition to the primitives needed to express the above patterns, Knative +Eventing defines two [_interface contracts_](#interface-contracts) to allow +connecting multiple types of Kubernetes objects as event senders and recipients +to the core primitives. + +![Overview of objects](images/eventing-overview.svg) + + + +## Interface Contracts + +In addition to the concrete types described below in the `messaging.knative.dev` +and `eventing.knative.dev` API groups, Knative Eventing supports referencing +objects in other API groups as destinations for event delivery. This is done by +defining partial schemas which the referenced resources must support. The +following interface contracts define resource fragments and partial schemas +(required fields on an arbitrary API object) which form a basis for Knative +Eventing. + +### Addressable + +[**Addressable**](./control-plane.md#addressable-v1) resources expose a resource +`address` (HTTP URL) in their `status` object. The URL is used as a destination +for delivery of events to the resource; the exposed URL must implement the +[data plane contract](data-plane.md) for receiving events. + +[**Broker**](#broker) and [**Channel**](#channel) both implement **Addressable** +to receive events from other components. + +### Destination + +[**Destination**](./control-plane.md#duckv1destination) is an interface +(resource fragment) which is used consistently through Knative Eventing to +reference an event delivery destination. A Destination eventually resolves the +supplied information to an URL, and may be an absolute URL or +[relative](https://datatracker.ietf.org/doc/html/rfc3986#section-4.2) to an +**Addressable** object reference; it also supports a Kubernetes Service object +reference (as a special case). An absolute URL in a Destination may be used to +reference cluster-external resources such as a virtual machine or SaaS service. + +### Event Source + +**Event Sources** are resources which generate events and may be configured to +deliver the events to a **Destination** designated by a `sink` object in the +resource's `spec`. The Knative Eventing spec does not define any specific event +sources, but does define common interfaces for discovering and managing event +sources. + +## Eventing + +### Broker + +[**Broker**](./control-plane.md#broker-v1) provides a central event-routing hub +which exposes a URL address which event senders may use to submit events to the +broker. A Broker may be implemented using many different underlying +event-forwarding mechanisms; the broker defines a small set of common +event-delivery configuration options and implementations of the broker may +define additional configuration options via a reference to an external object +(either a kubernetes built-in like ConfigMap or a custom object); the format of +the external objects is intentionally not standardized. + +### Trigger + +[**Trigger**](./control-plane.md#trigger-v1) defines a filtered delivery option +to select events delivered to a **Broker** and route them to a **Destination**. +Trigger implements uniform event filtering based on the CloudEvents attributes +associated with the event, ignoring the payload (which might be large and/or +binary and need not be parsed during event routing). The destination interface +contract allows Triggers to deliver events to either cluster-local objects or +external resources. + +## Messaging + +### Channel + +[**Channel**](./control-plane.md#channel-v1) provides an abstract interface +which may be fulfilled by several concrete implementations of a backing +asynchronous fan-out queue. The common abstraction provided by channel allows +both the composition of higher-level constructs for chained or parallel +processing of events, and the replacement of particular messaging technologies +(for example, allowing a development environment to use a lower-reliability +channel compared with the production environment). + +### Subscription + +[**Subscription**](./control-plane.md#subscription-v1) defines a delivery +destination for all events sent to a **Channel**. Events sent to a channel are +delivered to _each_ subscription _independently_ -- a subscription maintains its +own list of undelivered events and will manage retries independently of any +other subscriptions to the same channel. Like **Trigger**, subscriptions use the +**Destination** interface to support event delivery to many different +destination types.