Add ingress class docs

docs/README.md

@@ -7,6 +7,7 @@
   - [Custom Resources](#custom-resources)
   - [Deployment methods](#deployment-methods)
   - [High-availability and scaling](#high-availability-and-scaling)
+  - [Resource classes](#resource-classes)
   - [Security](#security)
 - [Guides and Tutorials](#guides-and-tutorials)
 - [Configuration reference](#configuration-reference)
@@ -48,6 +49,12 @@ and infrastructure.
 Please refer to [this document](concepts/ha-and-scaling.md) to understand
 failures scenarios, recovery methods, as well as scaling considerations.
 
+### Ingress classes
+
+[Ingress classes](concepts/ingress-classes.md) filter which resources the
+controller loads. They ensure that Kong Ingress Controller instances do not
+load configuration intended for other instances or other ingress controllers.
+
 ### Security
 
 Please refer to [this document](concepts/security.md) to understand the

docs/concepts/custom-resources.md

@@ -64,7 +64,7 @@ Once this resource is created, the resource needs to be associated with an
 `Ingress`, `Service`, or `KongConsumer` resource in Kubernetes.
 For more details, please read the reference documentation on `KongPlugin`.
 
-The below diagram shows how the `KongPlugin` resource can be linked to an
+The below diagram shows how you can link `KongPlugin` resource to an
 `Ingress`, `Service`, or `KongConsumer`:
 
 |  |  |
@@ -73,6 +73,8 @@ The below diagram shows how the `KongPlugin` resource can be linked to an
 
 ## KongClusterPlugin
 
+_This resource requires the [`kubernetes.io/ingress.class` annotation](../README.md#resource-classes)._
+
 KongClusterPlugin resource is exactly same as KongPlugin, except that it is a
 Kubernetes cluster-level resources instead of being a namespaced resource.
 This can help when the configuration of the plugin needs to be centralized
@@ -87,12 +89,20 @@ KongClusterPlugin with the same name.
 
 ## KongConsumer
 
+_This resource requires the `kubernetes.io/ingress.class` annotation. Its value
+must match the value of the controller's `--ingress-class` argument, which is
+"kong" by default._
+
 This custom resource configures `Consumers` in Kong.
 Every `KongConsumer` resource in Kubernetes directly translates to a
 [Consumer][kong-consumer] object in Kong.
 
 ## TCPIngress
 
+_This resource requires the `kubernetes.io/ingress.class` annotation. Its value
+must match the value of the controller's `--ingress-class` argument, which is
+"kong" by default._
+
 This Custom Resource is used for exposing non-HTTP
 and non-GRPC services running inside Kubernetes to
 the outside world via Kong. This proves to be useful when

docs/concepts/ingress-classes.md

@@ -0,0 +1,183 @@
+# Kong Ingress Controller and ingress class
+
+
+## Table of Contents
+
+- [**Introduction**](#introduction)
+- [**Configuring the controller class**](#configuring-the-controller-ingress-class)
+- [**Loading resources by class**](#loading-resouces-by-class)
+- [**When to use a custom class**](#when-to-use-a-custom-class)
+- [**Legacy behavior**](#legacy-behavior)
+- [**Examples**](#examples)
+
+## Introduction
+
+The Kong Ingress Controller uses ingress classes to filter Kubernetes Ingress
+objects and other resources before converting them into Kong configuration.
+This allows it to coexist with other ingress controllers and/or other
+deployments of the Kong Ingress Controller in the same cluster: a Kong Ingress
+Controller will only process configuration marked for its use.
+
+## Configuring the controller ingress class
+
+The `--ingress-class` flag (or `CONTROLLER_INGRESS_CLASS` environment variable)
+specify the ingress class expected by the Kong Ingress Controller. By default,
+it expects the `kong` class.
+
+## Loading resources by class
+
+The Kong Ingress Controller translates a variety of Kubernetes resources into
+Kong configuration. Broadly speaking, we can separate these resources into two
+categories:
+
+- Resources that the controller translates directly into Kong configuration.
+- Resources referenced by some other resource, where the other resource is
+  directly translated into Kong configuration.
+
+For example, an Ingress is translated directly into a Kong route, and a
+KongConsumer is translated directly into a [Kong consumer](https://docs.konghq.com/latest/admin-api/#consumer-object). A Secret containing
+an authentication plugin credential is _not_ translated directly: it is only
+translated into Kong configuration if a KongConsumer resource references it.
+
+Because they create Kong configuration indenpendent of any other resources,
+directly-translated resources require an ingress class, and their class must
+match the class configured for the controller. Referenced resources do not
+require a class, but must be referenced by a directly-translated resource
+that matches the controller.
+
+### Adding class information to resources
+
+Most resources use a [kubernetes.io/ingress-class annotation][class-annotation]
+to indicate their class. There are several exceptions:
+
+- v1 Ingress resources have a dedicated `class` field.
+- Knative Services [use the class specifed][knative-class] by the
+  `ingress.class` key of the Knative installation's `config-network` ConfigMap.
+  You can optionally [override this on a per-Service basis][knative-override]
+  by adding a `networking.knative.dev/ingress.class` annotation to the Service.
+
+### Enabling support for classless resources
+
+Specifying a class is optional for some resources. Although specifying a class
+is recommended, you can instruct the controller to process resources without a
+class annotation using flags:
+
+- `--process-classless-ingress-v1beta1` instructs the controller to translate
+  v1beta1 Ingress resources with no class annotation.
+- `--process-classless-kong-consumer` instructs the controller to translate
+  KongConsumer resources with no class annotation.
+
+These flags are primarily intended for compatibility with older configuration
+(Kong Ingress Controller before 0.10 had less strict class
+requirements, and it was common to omit class annotations). If you are creating
+new configuration and do not have older configuration without class
+annotations, recommended best practice is to add class information to Ingress
+and KongConsumer resources and not set the above flags. Doing so avoids
+accidentally creating duplicate configuration in other ingress controller
+instances.
+
+These flags do not _ignore_ `ingress.class` annotations: they allow resources
+with no such annotation, but will not allow resource that have a non-matching
+`ingress.class` annotation.
+
+## When to use a custom class
+
+Using the default `kong` class is fine for simpler deployments, where only one
+Kong Ingress Controller instance is running in a cluster. Changing the class is
+typical when:
+
+- You install multiple Kong environments in one Kubernetes cluster to handle
+  different types of ingress traffic, e.g. when using separate Kong instances
+  to handle traffic on internal and external load balancers, or deploying
+  different types of non-production environments in a single test cluster.
+- You install multiple controller instances alongside a single Kong cluster to
+  separate configuration into different Kong workspaces (using the
+  `--kong-workspace` flag) or to restrict which Kubernetes namespaces any one
+  controller instance has access to.
+
+## Legacy behavior
+
+This overview covers behavior in Kong Ingress Controller version 0.10.0 onward.
+Earlier versions had a special case for the default class and a bug affecting
+custom classes:
+
+- When using the default `kong` class, the controller would always process
+  classless resources in addition to `kong`-class resources. When using a
+  non-default controller class, the controller would only process resources
+  with that class, not classless resources. Although this was by design, it was
+  a source of user confusion.
+- When using a custom controller class, some resources that should not have
+  required a class (because they were referenced by other resources)
+  effectively did require a class: while these resources were loaded initially,
+  the controller would not track updates to them unless they had a class
+  annotation.
+
+In versions 0.10.0+ you must instruct the controller to load classless
+resources, which is allowed (but not recommended) for either the default or
+custom classes. Resources referenced by another resource are always loaded and
+updated correctly regardless of which class you set on the controller; you do
+not need to add class annotations to these resources when using a custom class.
+
+## Examples
+
+Typical configurations will include a mix of resources that have class
+information and resources that are referenced by them. For example, consider
+the following configuration for authenticating a request, using a KongConsumer,
+credential Secret, Ingress, and KongPlugin (a Service is implied, but not
+shown):
+
+```yaml
+kind: KongConsumer
+metadata:
+  name: dyadya-styopa
+  annotations:
+    kubernetes.io/ingress.class: "kong"
+username: styopa
+credentials:
+- styopa-key
+
+---
+
+kind: Secret
+apiVersion: v1
+stringData:
+  key: bylkogdatomoryakom
+  kongCredType: key-auth
+metadata:
+  name: styopa-key
+
+---
+
+kind: Ingress
+apiVersion: extensions/v1beta1
+metadata:
+  name: ktonezhnaet
+  annotations:
+    kubernetes.io/ingress.class: "kong"
+    konghq.com/plugins: "key-auth-example"
+spec:
+  rules:
+  - http:
+      paths:
+      - path: /vsemznakom
+        backend:
+          serviceName: httpbin
+          servicePort: 80
+
+---
+
+apiVersion: configuration.konghq.com/v1
+kind: KongPlugin
+metadata:
+  name: key-auth-example
+plugin: key-auth
+```
+
+The KongConsumer and Ingress resources both have class annotations, as they are
+resources that the controller uses as a basis for building Kong configuration.
+The Secret and KongPlugin _do not_ have class annotations, as they are
+referenced by other resources that do.
+
+[class-annotation]: ../references/annotations.md#kubernetesioingressclass
+[knative-class]: ../guides/using-kong-with-knative.md#ingress-class
+[knative-override]: https://knative.tips/networking/ingress-override/

docs/deployment/admission-webhook.md

@@ -142,6 +142,8 @@ $ echo "apiVersion: configuration.konghq.com/v1
 kind: KongConsumer
 metadata:
   name: harry
+  annotations:
+    kubernetes.io/ingress.class: kong
 username: harry" | kubectl apply -f -
 kongconsumer.configuration.konghq.com/harry created
 ```
@@ -153,6 +155,8 @@ $ echo "apiVersion: configuration.konghq.com/v1
 kind: KongConsumer
 metadata:
   name: harry2
+  annotations:
+    kubernetes.io/ingress.class: kong
 username: harry" | kubectl apply -f -
 Error from server: error when creating "STDIN": admission webhook "validations.kong.konghq.com" denied the request: consumer already exists
 ```

docs/guides/cert-manager.md

@@ -124,6 +124,8 @@ apiVersion: extensions/v1beta1
 kind: Ingress
 metadata:
   name: demo-yolo42-com
+  annotations:
+    kubernetes.io/ingress.class: kong
 spec:
   rules:
   - host: demo.yolo42.com
@@ -190,6 +192,7 @@ metadata:
   annotations:
     kubernetes.io/tls-acme: "true"
     cert-manager.io/cluster-issuer: letsencrypt-prod
+    kubernetes.io/ingress.class: kong
 spec:
   tls:
   - secretName: demo-yolo42-com

docs/guides/configure-acl-plugin.md

@@ -54,6 +54,7 @@ metadata:
   name: demo-get
   annotations:
     konghq.com/strip-path: "false"
+    kubernetes.io/ingress.class: kong
 spec:
   rules:
   - http:
@@ -71,6 +72,7 @@ metadata:
   name: demo-post
   annotations:
     konghq.com/strip-path: "false"
+    kubernetes.io/ingress.class: kong
 spec:
   rules:
   - http:
@@ -161,6 +163,7 @@ metadata:
   annotations:
     plugins.konghq.com: app-jwt
     konghq.com/strip-path: "false"
+    kubernetes.io/ingress.class: kong
 spec:
   rules:
   - http:
@@ -179,6 +182,7 @@ metadata:
   annotations:
     plugins.konghq.com: app-jwt
     konghq.com/strip-path: "false"
+    kubernetes.io/ingress.class: kong
 spec:
   rules:
   - http:
@@ -234,6 +238,8 @@ apiVersion: configuration.konghq.com/v1
 kind: KongConsumer
 metadata:
   name: admin
+  annotations:
+    kubernetes.io/ingress.class: kong
 username: admin
 " | kubectl apply -f -
 
@@ -242,6 +248,8 @@ apiVersion: configuration.konghq.com/v1
 kind: KongConsumer
 metadata:
   name: plain-user
+  annotations:
+    kubernetes.io/ingress.class: kong
 username: plain-user
 " | kubectl apply -f -
 ```
@@ -299,6 +307,8 @@ apiVersion: configuration.konghq.com/v1
 kind: KongConsumer
 metadata:
   name: admin
+  annotations:
+    kubernetes.io/ingress.class: kong
 username: admin
 credentials:
   - app-admin-jwt
@@ -309,6 +319,8 @@ apiVersion: configuration.konghq.com/v1
 kind: KongConsumer
 metadata:
   name: plain-user
+  annotations:
+    kubernetes.io/ingress.class: kong
 username: plain-user
 credentials:
   - app-user-jwt
@@ -536,6 +548,8 @@ apiVersion: configuration.konghq.com/v1
 kind: KongConsumer
 metadata:
   name: admin
+  annotations:
+    kubernetes.io/ingress.class: kong
 username: admin
 credentials:
   - app-admin-jwt
@@ -547,6 +561,8 @@ apiVersion: configuration.konghq.com/v1
 kind: KongConsumer
 metadata:
   name: plain-user
+  annotations:
+    kubernetes.io/ingress.class: kong
 username: plain-user
 credentials:
   - app-user-jwt
@@ -567,6 +583,7 @@ metadata:
   annotations:
     plugins.konghq.com: app-jwt,plain-user-acl
     konghq.com/strip-path: "false"
+    kubernetes.io/ingress.class: kong
 spec:
   rules:
   - http:
@@ -585,6 +602,7 @@ metadata:
   annotations:
     plugins.konghq.com: app-jwt,admin-acl
     konghq.com/strip-path: "false"
+    kubernetes.io/ingress.class: kong
 spec:
   rules:
   - http:

docs/guides/configuring-fallback-service.md

@@ -56,6 +56,7 @@ metadata:
   name: demo
   annotations:
     konghq.com/strip-path: "true"
+    kubernetes.io/ingress.class: kong
 spec:
   rules:
   - http:
@@ -104,6 +105,8 @@ apiVersion: extensions/v1beta1
 kind: Ingress
 metadata:
   name: fallback
+  annotations:
+    kubernetes.io/ingress.class: kong
 spec:
   backend:
     serviceName: fallback-svc