API Gateway Operator consists of two controllers that reconcile different CRs. To understand the reasons for using a single operator with multiple controllers instead of multiple operators, refer to the Architecture Decision Record. API Gateway Operator has a dependency on Istio and Ory Oathkeeper, and it installs Ory Oathkeeper itself.
Oathkeeper Deployment configures PodAntiAffinity to ensure that its Pods are evenly spread across all nodes and, if possible, across different zones. This guarantees High availability (HA) of the Ory Oathkeeper installation.
The following diagram illustrates the APIRule reconciliation process and the resources created in the process:
APIGateway Controller is a Kubernetes controller, which is implemented using the Kubebuilder framework. The controller is responsible for handling the APIGateway CR.
APIGateway Controller reconciles the APIGateway CR with each change. If you don't make any changes, the reconciliation process occurs at the interval of 10 hours.
APIGateway Controller reconciles only the oldest APIGateway CR in the cluster. It sets the status of other CRs to Warning.
If a failure occurs during the reconciliation process, the default behavior of the Kubernetes controller-runtime is to use exponential backoff requeue.
Before deleting the APIGateway CR, APIGateway Controller first checks if there are any APIRule or Istio Virtual Service resources that reference the default Kyma Gateway kyma-system/kyma-gateway. If any such resources are found, they are listed in the logs of the controller, and the APIGateway CR's status is set to Warning to indicate that there are resources blocking the deletion. If there are existing Ory Oathkeeper Access Rules in the cluster, APIGateway Controller also sets the status to Warning and does not delete the APIGateway CR.
The gateways.operator.kyma-project.io/api-gateway-reconciliation finalizer protects the deletion of the APIGateway CR. Once no more APIRule and VirtualService resources are blocking the deletion of the APIGateway CR, the APIGateway CR can be deleted. Deleting the APIGateway CR also deletes the default Kyma Gateway.
APIRule Controller is a Kubernetes controller, which is implemented using the Kubebuilder framework.
The controller is responsible for handling the APIRule CR.
Additionally, the controller watches the api-gateway-config to configure the JWT handler.
APIRule Controller has a conditional dependency to APIGateway Controller in terms of the default APIRule domain. If you don't configure any domain in APIGateway CR, APIRule Controller uses the default Kyma Gateway domain as the default value for creating VirtualServices.
NOTE: For now, you can only use the default domain in APIGateway CR. The option to configure your own domain will be added at a later time. See the epic task.
APIRule Controller reconciles APIRule CR with each change. If you don't make any changes, the process occurs at the default interval of 30 minutes. You can use the API Gateway Operator parameters to adjust this interval. In the event of a failure during the reconciliation, APIRule Controller performs the reconciliation again after one minute.
The following diagram illustrates the reconciliation process of APIRule and the created resources:
The APIRule reconciliation supports different processors that are responsible for validation and status handling as well as creating, updating, and deleting the resources in the cluster.
The processor used is evaluated for each reconciliation of an APIRule and is determined by the configuration of the JWT handler in the api-gateway-config ConfigMap or the existence of the
annotation gateway.kyma-project.io/original-version: v2alpha1 on the APIRule.
The processor is selected based on the following rules:
- If the handler in the
api-gateway-configConfigMap is set toistio, the APIRule reconciliation uses theNewIstioReconciliationin the istio package. - If the handler in the
api-gateway-configConfigMap is set toory, the APIRule reconciliation uses theNewOryReconciliationin the ory package. - If the annotation
gateway.kyma-project.io/original-version: v2alpha1is present on the APIRule, the APIRule reconciliation uses theNewReconciliationin the v2alpha1 package.
Certificate Controller is a Kubernetes controller, which is implemented using the Kubebuilder framework.
The controller is responsible for handling the Secret api-gateway-webhook-certificate in the kyma-system namespace. This Secret contains the Certificate data required for the APIRule conversion webhook.
Certificate Controller reconciles a Secret CR with each change. If you don't make any changes, the process occurs at the default interval of 1 hour. This code verifies whether the Certificate is currently valid and will not expire within the next 14 days. If the Certificate does not meet these criteria, it is renewed. In the event of a failure during the reconciliation, Certificate Controller performs the reconciliation again with the predefined rate limiter.
RateLimit Controller is a Kubernetes controller, which is implemented using the Kubebuilder framework. The controller is responsible for handling the RateLimit CR.
RateLimit Controller reconciles the RateLimit CR with each change. If you don't make any changes, the process occurs at the default interval of 30 minutes. In the event of a failure during the reconciliation, RateLimit Controller performs the reconciliation again with the predefined rate limiter.