Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GEP-1282 Backend Properties - Update implementation #1430

Closed
wants to merge 4 commits into from
Closed

GEP-1282 Backend Properties - Update implementation #1430

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
b327b15
GEP-1282 Backend Properties - Update implementation
candita Oct 3, 2022
67366d5
101022 First round of changes from feedback
candita Oct 10, 2022
db7b3f3
101022 Second round of changes from community meeting feedback
candita Oct 11, 2022
07bf512
GEP-1282 Backend Properties -focus the fields in on the things a clie…
candita Oct 17, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
298 changes: 293 additions & 5 deletions site-src/geps/gep-1282.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ The purpose of this GEP is to add APIs for capturing those backend capabilities
As implementations have started to build out HTTPRoute support, requests for some common capabilities have started coming up, particularly TLS re-encryption (encryption between Gateway and backend), Websockets tracking, and HTTP/2 support.

Evan Anderson opened a [discussion that generated a lot of interest](https://github.com/kubernetes-sigs/gateway-api/discussions/1244), and during that discussion, we brought up a few more points:

* Whatever we do to solve this in the HTTPRoute case may be applicable describing more general service properties, like identity, which could be useful for other Route use cases.
* This may be very useful for mesh/GAMMA use cases as well as more generally for adding arbitrary future capabilities to the backend service. An example is a CA cert for connecting to the backend - that’s tightly bound to the service, but there’s nowhere to put it at the moment.

Expand All @@ -46,25 +47,30 @@ This initial form of this GEP is for the Gateway API community to agree on what
### Why build something?

We've got the following feature requests and discussions in the Gateway API repo:
- [#1244](https://github.com/kubernetes-sigs/gateway-api/discussions/1244) : Unclear how to specify upstream (webserver) HTTP protocol. This issue describes the problems that Evan had in trying to be able to define if a backend behind a HTTPRoute supports HTTP2 over cleartext or websockets. The question of how to tell the Gateway implementation that the backend needs TLS details for a proxy-based implementation to be able to connect it also came up in the discussion.
- [#1285](https://github.com/kubernetes-sigs/gateway-api/discussions/1285) has a more specific discussion about how different ingress implementations allow this to be configured today, whether that's with the Ingress resource or their own custom one. The great roundup that Candace did is reproduced in the next few bullet points.
* [#1244](https://github.com/kubernetes-sigs/gateway-api/discussions/1244): Unclear how to specify upstream (webserver) HTTP protocol. This issue describes the problems that Evan had in trying to be able to define if a backend behind a HTTPRoute supports HTTP2 over cleartext or websockets. The question of how to tell the Gateway implementation that the backend needs TLS details for a proxy-based implementation to be able to connect it also came up in the discussion.

* [#1285](https://github.com/kubernetes-sigs/gateway-api/discussions/1285) has a more specific discussion about how different ingress implementations allow this to be configured today, whether that's with the Ingress resource or their own custom one. The great roundup that Candace did is reproduced in the next few bullet points.

* Istio uses a [DestinationRule resource with ClientTLSSettings](https://istio.io/latest/docs/reference/config/networking/destination-rule/#ClientTLSSettings) to capture TLS details, and the DestinationRule resource also holds traffic policy information like load balancing algorithm, connection pool size, and so on.
* Openshift’s Route resource allows the [configuration of reencryption](https://docs.openshift.com/container-platform/4.10/networking/routes/secured-routes.html#nw-ingress-creating-a-reencrypt-route-with-a-custom-certificate_secured-routes) specifically, along with custom certificate details.
* Contour’s HTTPProxy captures TLS details using an Envoy client certificate, destination CA certificate, and optional SubjectName which sets what Envoy should expect to see from the backend service, all inside the HTTPProxy resource. It also requires either a Protocol field inside the HTTProxy, or an annotation on the Service that tells Contour that the service expects TLS. This is [all documented](https://projectcontour.io/docs/v1.21.1/config/upstream-tls/), but I should note that Contour’s docs use the Envoy convention where a backend in Gateway parlance is called an Upstream (which may be confusing if you’re not used to it).
* Linkerd uses a [Server resource](https://linkerd.io/2.11/reference/authorization-policy/#server) (which is functionally pretty similar to Service in that it associates a name with a Pod selector, but also has other details like if the service supports Proxy protocol), along with a ServerAuthorization resource that specifies some constructs that sit more at the service mesh level, including identity and access control.

In terms of other implementations existing use cases for features like this:

- For Contour, there are annotations to allow the configuration of the following on a backend Service:
- Max connections
- Max pending requests
- Max requests
- Max retries
- Upstream protocol: This is what allows Contour to handle switching protocols for the backend service.

- In Istio, the `DestinationRule` resource allows the configuration of many settings like this:
- Load balancer algorithms
- Connection pool settings
- Outlier Detection
- Tunnel settings for non-HTTP workloads

- In Consul, [`ServiceDefaults`](https://www.consul.io/docs/connect/config-entries/service-defaults) allows specifying similar configuration:
- Max connections
- Max pending requests
Expand All @@ -74,6 +80,7 @@ In terms of other implementations existing use cases for features like this:
- TLS SNI (we don't support configuring a specific CA at this granularity, but publishing a public key in a status field could be useful)

The properties we're talking about all share two things:

- They are tightly bound to the backend, rather than being important at the route level
- And the service owner (application developer in the Gateway API personas) should control these settings, not the owner of the Gateway implementation (the Cluster Operator)

Expand All @@ -90,6 +97,7 @@ We’re looking to add specific, structured extension points somewhere in the re
Those specific, structured extension points need to be in a place where they can be owned by the person who owns the backend, since that could be different to the person who owns the Route. That is, whatever we choose should be something that extends a thing that is referred to by a `backendRef` in a Route, not inside the Route.

The initial list includes, but is not limited to:

* TLS information for connection from the Gateway to the backend service. Note that this doesn’t include any information used for service mesh encryption, just what a Gateway’s proxy would need to be able to connect to the backend service.
* Websocket protocol information for the backend service.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think mixing TLS info with protocol info is ideal. TLS is pretty advanced and scary for many users.
Also in many cases this is negotiated at protocol level, not config ( ALPN, etc)

* Protocol disambiguation for “upgradeable” protocols like HTTP/2 and HTTP/1.1 which operate on the same port. (Websockets may be another case of this.)
Expand All @@ -98,12 +106,292 @@ Of course, from the section above, you can see that there are many other feature

## API
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mentioned this on the call but want to re-iterate so its in writing: I think BackendCapabilities is the wrong abstraction for this API.

What BackendCapabilities says is "Here are the properties of the backend". This may be something like "Listens on TLS1.2 or TLS1.3 over HTTP/2" or "Supports HTTP/1.1 and Http/2", etc.

What we want to do, as implementors, is program a client to decide how to connect. With the information in this API, we don't have enough data to know how to program a client.

If a backend says they support HTTP/1.1 and HTTP/2, how do I know which one to send?

If a backend says it supports TLS, I have the same problem. As a gateway, I can either forward a request as is or add a layer of TLS. But in many cases, we won't actually know if the original request already is TLS encrypted, so we don't know whether we should add a layer of TLS or not.

Ultimately, this configuration is a consumer configuration; by treating it as a producer configuration we are not getting the functionality required to implement this effectively.

Note: you may look at DestinationRule and make the same argument, but actually it is a consumer configuration that allows the producer to set defaults for consumers; for reasons mentioned above, setting a default policy of "TLS encrypt" would almost certainly break users.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @howardjohn for writing this out, I understand the concern better.

My viewpoint, and I think @youngnick 's, is that by creating BackendProperties that reference a backend, the developer or admin is telling the Gateway how to interact with the backend. The intention is still to use BackendCapabilities as a wrapper or decorator for the backend.

I may miss some nuances here, but I thought what might happen is that the Gateway (acting in its client role) would lookup and use BackendCapabilites as its reference to the backend. For a HTTPRoute that is not a passthrough (assuming the Gateway knows which traffic is passthrough), if the Gateway finds a BackendCapabilitesSpec for the backend, then it uses that information to decide how to connect. If it doesn't find a spec, then it connects as it would have without knowing the backend capabilities.

Note I started with BackendTargetRef as a part of BackendCapabilities, not BackendCapabilitesSpec. Later I moved it into the spec by request from @robscott (#1430 (comment)). Does hiding the BackendTargetRef within BackendCapabilitesSpec make programming a client more difficult?

Is there any way to salvage the BackendCapabilies as an abstraction? We can definitely add more to make it work properly if it is still missing too much. Maybe we need to store more of the handshake information for TLS, there are a few pieces missing, like details on cipher suite, compression, curve, etc. And there are few pieces that still need tweaking.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@howardjohn, there are a few reasons why we're doing this with a separate resource:

  • To avoid adding fields to the backendRef that may not always make sense there
  • To avoid having to set fields individually for each backendRef - this cost increases for each time a backend is referenced in a Route config.
  • To make it so that this config is available to other things than HTTPRoute without reimplementing in every Route type

The basic idea is that there's an object, owned by the owner of the Service that tells consumers what they can do when they connect to the service. You're one hundred percent right that these are consumer rules, but the only person who knows what these rules should be is the person who sets up the Service - the Producer.. So this is a way for the service owner/producer to tell Service consumers about how to connect to the service, in a structured way.

It seems to me like having the Route/Gateway need to guess at these rules is not ideal, unless we assume that the Route owner always is the Service producer.

If that is the case, then we can easily do this by adding fields to backendRef. The tradeoff there is that now, everywhere that uses backendRef needs to have rules about what happens when you specify TLS settings, or websocket settings, or whatever. So if you use your backend in more than one backendRef, you now need to duplicate all these settings.

Copy link

@tsaarni tsaarni Oct 13, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the problem just that calling this "backend properties/capabilities" is guiding towards wrong abstraction? For example, the description currently in the TLS fields says:

    // The minimal TLSVersion that is accepted by the backend.
    ...
    // CACerts are the Certificate Authority, trusted by both Gateway and backend,

This wording makes sense if one would really describe capabilities of the backend. But since we are programming the client/gateway/proxy, speculating about the backend capabilities seems bit unexpected. We end up deriving client configuration from server capabilities?

What I was expecting to see was simply properties applied to Gateway's backend connection(s) when communicating towards specific backend(s). Something like "the minimum TLSVersion used by the Gateway when connecting to the backend" and "Certificate Authorities used by Gateway to verify backend server certificate". I'm not sure if the same holds to everything, but at least for backend TLS hop it seems so.

Edit: If one would use this same abstraction to configure cluster external properties as well (such as letting backend developer to provision external server certs #763), then "backend connection properties" would be misleading name as well.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The alternative is not to make every use in backendRef. The core relationship of consumer + producer rules, IMO, is "Producer sets defaults, can be overriden by consumer" (which is essentially what policy attachment is all about, as well). Its not necessarily either-or.

For a HTTPRoute that is not a passthrough (assuming the Gateway knows which traffic is passthrough), if the Gateway finds a BackendCapabilitesSpec for the backend, then it uses that information to decide how to connect.

Focusing overly on HTTPRoute (here and in other GEPs, we tend to) is not always the best path. What if the route is a TCPRoute? The contents are opaque - we cannot tell if its TLS or not.

Even if its a TLSRoute - you know it has a layer of TLS, but how many? How many does the backend expect? May sound crazy, but its not uncommon to have multiple layers of TLS at this point with transparent mTLS, things like onion routing, etc.

But most importantly - backendProperties, as proposed, are a property of the backend. That means there is no relationship to Gateway or Routes at all. These properties should be respected by any Kubernetes cluster that can support them, not just N/S traffic (mesh).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I was expecting to see was simply properties applied to Gateway's backend connection(s) when communicating towards specific backend(s). Something like "the minimum TLSVersion used by the Gateway when connecting to the backend" and "Certificate Authorities used by Gateway to verify backend server certificate". I'm not sure if the same holds to everything, but at least for backend TLS hop it seems so.

Is that not what this proposal is doing? Is it just the name BackendCapabilities that is the sticking point? The idea here is to allow the owner of the service to say "When connecting to the thing that's running inside the pod. do these things" - have a minimum TLS version, use this CA cert and so on. If it's just that the word Capabilities is too overloaded, and we need to change the name, I don't think that's a big deal.

@tsaarni, in the quote you've given, I think it's equally correct to have that say:

    // The minimal TLSVersion required to connect to the backend.
    ...
    // CACerts are the Certificate Authority, that the serving certificate for the backend service must be validated against.

Both of those properties are things that are set by the owner of the Service, but are needed by clients that connect to the Service. Similarly, the point of having websocket details in a resource like this is so that the owner of the Service can say "My service will accept websockets at the following paths", so that clients (like a Gateway proxy) know what to do for those paths.

Again, the point here is to take a whole bunch of configuration that, in the Ingress world, was often done via annotations on the Service, and make it structured. Ideally, we would put this inside the Service in structured fields, but we can't easily change the Service spec, so we've opted for a "decorator" style CRD.

If it's just that the name BackendCapabilities is creating the wrong abstraction, that's fine, but we need some guidance around what else to call it.

@howardjohn, I'm not quite sure I understand what you're asking for. Are you saying that this resource should be called BackendHTTPProperties or something, and only be relevant when it's used as a backendRef in a HTTPRoute? Or are you saying that we should define semantics for the other Route types?

It's clear that you disagree with the current approach, but I am not sure what actions we need to take to move towards something we can agree on.

Copy link
Contributor

@howardjohn howardjohn Oct 14, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consumers need to be able to control the settings, not just the service producers.

I am definitely not suggesting making it tied to HTTP - my concern was the opposite. That this won't work for non-http clients because we don't know if we should encrypt or not. But the client knows which is why they need to be able to configure it

Copy link
Contributor

@youngnick youngnick Oct 14, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, I agree that consumers also need a place for some settings like this. But don't we need to have the service producer's settings available at least, and then we can see what things make sense to allow in other places? I'll be honest, in my head I kind of thought that Policy Attachment could be used to set things from the Gateway side - so the producer can say "I accept TLS 1.2 or greater", and a Policy could say "All Routes attached to this Gateway will default to TLS 1.3" and then things would work. But without the bit where the producer says what they expect, then you can't even do simple cases, where there's no extra TLS wrapping etc.

If this needs explanation in the document as a place for future work, I think that's fine. But I don't know how we solve the basic problem of "My service is doing its own TLS, and proxy connections from a Gateway won't work unless it knows about it" without something like what we have here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with you but I don't think the GEP behaves that way? Or I read it wrong

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we have anything in the GEP right now about Policy Attachment and consumer settings, but that's probably worth adding.

I'll review the GEP again and see if I can see any other changes, and then come back here.


To be written later, once we have agreement on the “what” and the “why”. (This is the “how”).

A new object, BackendCapabilities, is added to the API that serves as a wrapper to a Kubernetes Service, and provisions
backend capabilities for a Service that requires them. As discussed by the Gateway API maintainers and community, this
architectural choice is made because we would ideally like to locate this capabilities set within the Service but until
that is possible in Kubernetes, we can start with a wrapper to the Service.

BackendCapabilities contains a BackendCapabilitiesSpec object that specifies the targeted backend and defines each of
the different backend capabilities. In this GEP, TLS encryption between gateway and backend is proposed as a capability,
but with a design that allows additional capabilities to be added as needed in subsequent versions of
Gateway API. BackendCapabilities also contains a BackendCapabilitiesStatus which holds the current state.

In https://github.com/kubernetes-sigs/gateway-api/blob/main/apis/v1beta1/shared_types.go add a new type:
```go
// BackendCapabilities references a Kubernetes Service and defines any backend
// capabilities that Service requires.
//
// +optional
type BackendCapabilities struct {
// Name is the name of this backend capability set.
// +required
Name string `json:”name”`

// Spec specifies the capabilities supported by this BackendCapabilities
// Service wrapper, if any. The capabilities are predefined, and have two
// characteristics in common: they are tightly bound to the backend,
// rather than being important at the route level, and the service owner
// (application developer in the Gateway API personas) should control
// these settings, not the owner of the Gateway implementation (the
// Cluster Operator in the Gateway API personas).
//
// Support: Extended
//
// +required
Spec BackendCapabilitiesSpec `json:”spec”`

// Status represents the current state of the backend and its capabilities.
// +optional
Status *BackendCapabilitiesStatus `json:”status,omitempty”`
}

// BackendTargetReference identifies an API object to apply BackendCapabilities to.
//
type BackendTargetReference struct {
// Group is the group of the target resource.
Group Group `json:"group"`

// Kind is kind of the target resource.
// Valid values include:
// * "Service" (the default if no Kind is specified)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How can it be a Service if it's a backend/workload capability ? Deployment/ReplicaSet/etc - or pod label selector maybe ?

If you want this to be a Service extension - call it ServiceProperties or ServiceTLS, etc.

At least in Istio we make a very big distinction between workloads - pods selected by label - and Services. I think many users treat Service as a frontend - and the selected Pods as backend.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is absolutely the Service being used as a backend, as a shortcut for a selector. That's why we called it _Backend_Capabilities, not just _Service_Capabilities. Remember, a Service is both the frontend (what you spend most of your time thinking of it as, because that's what Istio needs), and the backend, a bucket of workload endpoints (which is how people who aren't doing service mesh tend to see it more).

Having this reference a Service resource is a way avoid having to store the workload selector in two places, as well as a way to meet people who are using Ingress controllers for north-south traffic more than they are Services for east-west traffic.

// * "ServiceImport"
//
// Invalid values include:
//
// * "invalid/kind" - "/" is an invalid character
// +required
Kind string `json:"kind"`

// Name is the name of the target resource.
Name ObjectName `json:"name"`
}

// BackendCapabilitiesSpec defines a group of specific properties held by its
// affiliated backend, and specified by the backend owner (typically the
// application developer).
//
// Examples of properties that belong in the BackendCapabilitiesSpec include:
// TLS re-encryption, session state, timeouts, retries, Websockets, HTTP/2,
// load balancer algorithm, healthcheck info, external traffic policy,
// connection limits, error page, protocol switch, alerts, and others.
//
// Support: Extended
//
type BackendCapabilitiesSpec struct {
// Backend refers to the object to which we apply these capabilities
// +required
Backend backendTargetReference `json:"backend,omitempty"`

// TLS is the section of details on encryption processing in the traffic
// hop from Gateway to backend, sometimes known as upstream TLS or re-encrypt.
// A xRoute configured as passthrough cannot specify a TLS encrypt type.
// TLS is useful for HTTPRoute and should not be used on UDP/TCP/TLSRoute.
//
// +optional
TLS *EncryptType `json:"tls,omitempty"`
}

// EncryptType describes the details needed by the client (Gateway) to implement
// encryption from the client to the destination endpoint.
type EncryptType struct {
// The minimal TLSVersion that is accepted by the endpoint.
// If empty, there is no minimal backend TLSVersion.
// +optional
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know if this is something each individual service owner should mess with and configure individually.

TLS compliance is an admin choice, may be configured on the Gateway or on cluster or implementation-specific config. In most cases we want to prevent service owners to downgrade security or break the compliance requirements by missconfiguration.

This is even a bigger problem for CipherSuites - do we expect each individual backend owner to understand what cipher suites are safe and configure them ? This is something very few people know or should mess with.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think maybe there's some confusion - I don't think I've ever intended this to be the only way that TLS will be enabled to backends. This construct is to cover when the service owner wants to manage their own TLS, otherwise the Gateway would be able to contact the service directly. (I also include in that case the mesh-style magic infrastructure that will transparently encrypt connections for you).

Right now, there is no way to represent the (pretty common) case of someone saying "my service is running its own TLS, I need it exposed via a Gateway". That's the primary use case we're trying to solve here - the extra applicatibility is useful but not essential.

I guess another way to say this is - regardless of what service owners should be doing, they absolutely do want to do what this object is providing, already.

MinVersion *TLSVersion `json:"minVersion,omitempty"`

// The maximum TLSVersion that is accepted by the endpoint.
// If empty, there is no maximum backend TLSVersion.
// +optional
MaxVersion *TLSVersion `json:"maxVersion,omitempty"`

// CipherSuites is the minimum cipher suite that the backend accepts.
// If empty, anything that is appropriate for the MinVersion is accepted.
// If the MinVersion is not specified, the result is implementation
// dependent, e.g. based on the minimimal TLS version supported by the
// implementation.
// +optional
CipherSuites []string `json:"cipherSuites,omitempty"`

// CACerts is the PEM bundle containing Certificate Authority, trusted by both
// client and endpoint, and intermediate certificates needed to complete the
// certificate chain of trust. If empty, the system root certs should be used.
//
// +optional
CACerts *SecretObjectReference `json:"caCerts,omitempty"`
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's the biggest problem - we can't have a SecretObjectReference crossing a namespace boundary. We had CVEs related to this, it's not good.

Also not sure what it means 'trusted by both' - I suspect what you want to define here is the root CA in case the backend is using a private root. SPKI or the public key or cert - inline, like it is used in kubeconfig - may work.
However I don't know if this API is the proper place to define root key managed and distribution - there is a working
group I think working on certs and security.

I don't disagree that supporting serlf-signed or private root is important - just that PKI config is probably better as a separate concern, not part of backend properties - and can't be based on cross namespace Secret references ( or expectation that each namespace has a secret with that name).

Copy link
Contributor

@youngnick youngnick Oct 21, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This Secret reference is not for some other service to connect to this one, it's for the Gateway controller (which already needs cluster-wide access to secrets to be able to do its job). The main, relevant client here is the Gateway controller.

And even if users do need to do a cross-namespace Secret reference, that's exactly what ReferenceGrant is for, to enable the controlled use of Secrets or other resources across namespaces. It seems reasonable to assume that, for self-signed certs, a CA cert might live in its own namespace, with a ReferenceGrant then being required to allow access to it. Or, of course, a controller could just copy that into relevant namespaces too.

Lastly, this is a CA certificate chain, not a key, so it's intended to be public. We could use another object (maybe a configmap?) to store it if necessary.


// ServiceNames provide multiple named domains which will share a certificate
// for the endpoint.
// +optional
ServiceNames []string `json:"serviceNames,omitempty"`
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why ? How does it help the caller to know ahead of time the list that will be returned in the cert anyways ?

You mean 'SANNames' - for supporting spiffee or equivalent ? That may be useful.
Do you mean this as an override of the normal TLS server verification - where gateway talk with foo.ns.svc.cluster.local but get a certificate for example.com - I think it's important to be very clear with this, security is tricky.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this case, it's to help the Gateway implementation so that it can determine configuration validity before allowing any connections at all. I'm so-so on this one, because using it makes the hostname match a three-part one (Listener, HTTPRoute, BackendCapabilities), rather than a two-part one, which is already hard to do. The upside is that it means that implementations can easily check and make clear at a configuration level when something is not going to work, rather than waiting to have client requests fail and needing to troubleshoot that.

}

// BackendCapabilitiesStatus describes the status of the backend and its
// capabilities.
type backendCapabilitiesStatus struct {
// List of conditions that represent the current status of the
// the backend capabilities object.
// TODO: Define constants to represent typical status conditions
//
// +kubebuilder:validation:MinItems=1
// +kubebuilder:validation:MaxItems=8
Conditions []metav1.Condition `json:”conditions,omitempty"`
}
```

###Example Capabilities
For a Service `private-service` defined as:

```yaml
apiVersion: v1
kind: Service
metadata:
name: private-service
spec:
selector:
app.kubernetes.io/name: privateApp
ports:
- protocol: TCP
port: 80
targetPort: 9099
```

We could create a backendCapabilities for TLS re-encrypt on `private-service` defined as:

```yaml
apiversion: v1beta1
kind: backendCapabilities
metadata:
name: private-service-caps
spec:
backend:
kind: Service
name: private-service
TLS:
minVersion: 1.3
cipherSuites:
- TLS_AES_256_GCM_SHA384
- TLS_CHACHA20_POLY1305_SHA256
- TLS_AES_128_GCM_SHA256
caCerts: my-cert-bundle
serviceNames:
- private-service.example.com
- dev.example.com
```

Implementers of TLS re-encrypt would have to check for backendCapabilities that shared the same spec.backend.name as
the targeted Service, to discover the capabilities specification they would need to perform encryption between the
Gateway and the backend.

## Alternatives

Again, we need to wait until we’re writing the “how”.
### Alternative 1 - map of structs

One alternative merits a mention. In this alternative proposal, the API will be composed of several new objects:
BackendObjectCapabilities, an instance of BackendObjectCapabilitiesMap, defined as a mapping of capability name to an
array of BackendObjectCapability objects. BackendObjectCapability objects are simple but expressive Group-Kind-Name
objects with an additional Value field implemented as an interface so that it can store any type of data.

In https://github.com/kubernetes-sigs/gateway-api/blob/main/apis/v1beta1/shared_types.go:
```go
// BackendRef defines how a Route should forward a request to a Kubernetes
// resource.
//
// ...
type BackendRef struct {
// BackendObjectReference references a Kubernetes object.
BackendObjectReference `json:",inline"`

// BackendObjectCapabilities specifies the capabilities supported by this
// BackendRef, if any. The capabilities are not predefined, but have two
// characteristics in common: they are tightly bound to the backend,
// rather than being important at the route level, and the service owner // (application developer in the Gateway API personas) should control
// these settings, not the owner of the Gateway implementation (the
// Cluster Operator in the Gateway API personas).
//
// There are a maximum of 32 BackendObjectCapabilities per BackendRef.
//
// Support: Extended
//
// +optional
// +kubebuilder:validation:MaxItems=32
BackendObjectCapabilities *BackendObjectCapabilitiesMap `json:”backendObjectCapabilitiesMap,omitempty”`

// Weight specifies the proportion of requests forwarded to the referenced
//...
Weight *int32 `json:"weight,omitempty"`
}

// BackendObjectCapabilitiesMap defines a mapping of BackendObjectCapability name
// to an array of BackendObjectCapability
BackendObjectCapabilitiesMap map[string][]BackendObjectCapability

// BackendObjectCapability defines a capability or property held by its
// affiliated BackendObject, and specified by the BackendObject owner
// (typically the application developer). Examples of BackendObjectCapability
// use cases include: TLS re-encryption, session state, timeouts, retries,
// Websockets, HTTP/2, load balancer algorithm, healthcheck info, external
// traffic policy, connection limits, error page, protocol switch, alerts.
//
// The BackendObject must be valid in the cluster; the Group and Kind must
// be registered in the cluster for this reference to be valid.
//
// Support: Extended
//
// References to objects with invalid Group and Kind are not valid.
type BackendObjectCapability struct {
// Group is the group of the referent.
//
// Support: Core
//
// +kubebuilder:default=gateway.networking.k8s.io
// +optional
Group *Group `json:"group,omitempty"`

// Kind is kind of the referent.
//
// Support: Custom
//
// +kubebuilder:default=Gateway
// +optional
Kind *Kind `json:"kind,omitempty"`

// Name is the name of the referent.
//
// Support: Core
Name ObjectName `json:"name"`

// Value is the value of the referent. The type is not specifically
// defined and is left up to the implementation to infer.
//
// Support: Extended
//
// +optional
Value interface{} `json:”value”`
}

```
### Example Capabilities for Alternative 1

TLS information for connection from the Gateway to the backend service. What a Gateway’s proxy would need to be able to
connect to the backend service.

The BackendCapabilities could be defined by an implementation and look something like:

```yaml

backendCapabilities:
tls-reencrypt:
- name: version
value: 1.3
- name: clientCert
value: clientSecret
- name: privateKey
value: pkeySecret
- name: caCerts
value: cacertSecret
- name: backendCert
value: backendSecret
```

## References

Expand Down