-
Notifications
You must be signed in to change notification settings - Fork 497
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
created 231 aks apim self hosted gateway
- Loading branch information
Houssem Dellai
committed
Jul 21, 2024
1 parent
545b066
commit 9ecb3d3
Showing
42 changed files
with
1,013 additions
and
10 deletions.
There are no files selected for viewing
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+16.2 KB
231_aks_ingress_apim_gateway/kubernetes/apim-self-hosted-gateway.yaml
Binary file not shown.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,38 @@ | ||
apiVersion: v1 | ||
kind: Namespace | ||
metadata: | ||
name: backend | ||
--- | ||
apiVersion: apps/v1 | ||
kind: Deployment | ||
metadata: | ||
name: webapi | ||
namespace: backend | ||
spec: | ||
replicas: 3 | ||
selector: | ||
matchLabels: | ||
app: webapi | ||
template: | ||
metadata: | ||
labels: | ||
app: webapi | ||
spec: | ||
containers: | ||
- name: webapi | ||
image: ghcr.io/houssemdellai/containerapps-album-backend:v1 | ||
ports: | ||
- containerPort: 3500 | ||
--- | ||
apiVersion: v1 | ||
kind: Service | ||
metadata: | ||
name: webapi | ||
namespace: backend | ||
spec: | ||
type: ClusterIP | ||
ports: | ||
- port: 80 | ||
targetPort: 3500 | ||
selector: | ||
app: webapi |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,159 @@ | ||
# Exposing AKS applications using API Management | ||
|
||
## Introduction | ||
|
||
Mutual TLS authentication is natively supported by API Management and can be enabled in Kubernetes by installing an Ingress Controller. As a result, authentication will be performed in the Ingress Controller, which simplifies the microservices. Additionally, you can add the IP addresses of API Management to the allowed list by Ingress to make sure only API Management has access to the cluster. | ||
|
||
 | ||
|
||
Pros: | ||
|
||
- Easy configuration on the API Management side because it doesn't need to be injected into the cluster VNet and mTLS is natively supported | ||
- Centralizes protection for inbound cluster traffic at the Ingress Controller layer | ||
- Reduces security risk by minimizing publicly visible cluster endpoints | ||
|
||
Cons: | ||
|
||
- Increases complexity of cluster configuration due to extra work to install, configure and maintain the Ingress Controller and manage certificates used for mTLS | ||
- Security risk due to public visibility of Ingress Controller endpoint(s) | ||
|
||
When you publish APIs through API Management, it's easy and common to secure access to those APIs by using subscription keys. Developers who need to consume the published APIs must include a valid subscription key in HTTP requests when they make calls to those APIs. Otherwise, the calls are rejected immediately by the API Management gateway. They aren't forwarded to the back-end services. | ||
|
||
To get a subscription key for accessing APIs, a subscription is required. A subscription is essentially a named container for a pair of subscription keys. Developers who need to consume the published APIs can get subscriptions. And they don't need approval from API publishers. API publishers can also create subscriptions directly for API consumers. | ||
|
||
Official documentation here: https://learn.microsoft.com/en-us/azure/api-management/api-management-kubernetes | ||
|
||
## Lab: Exposing AKS applications using API Management | ||
|
||
In this lab, you will learn how to expose an AKS application using API Management. You will go through the following steps: | ||
|
||
1. Deploy an AKS cluster | ||
2. Enable AKS App Routing addon to create a managed Nginx Ingress Controller | ||
3. Deploy an application to the cluster | ||
4. Expose the application using the Ingress Controller and internal Load Balancer | ||
5. Deploy an API Management instance | ||
6. Expose the application using API Management | ||
|
||
### Deploying the resources | ||
|
||
You will use `terraform` to deploy the resources. | ||
|
||
To deploy the resources, run the following commands from the `terraform` directory: | ||
|
||
```sh | ||
terraform init | ||
terraform apply -auto-approve | ||
``` | ||
|
||
This will take about 22 minutes to complete. | ||
The following resources will be deployed:  | ||
|
||
- AKS cluster with App Routing addon enabled | ||
- API Management developer instance (the cheapest SKU) | ||
- Virtual Network with a subnet for the AKS cluster and another subnet for the API Management instance | ||
- NSG rules to allow required inbound and outbound traffic for API Management instance | ||
- API definition for the application in API Management | ||
|
||
> Note how we are using the static IP address `10.10.0.10` for the Ingress Controller. This IP will be used to configure the API Management API backend. | ||
After that, you will need to deploy an application to the AKS cluster. To do this, run the following commands from the `kubernetes` directory: | ||
|
||
```sh | ||
kubectl apply -f 1-app.yaml,2-nginx-internal-controller.yaml,3-ingress-internal.yaml | ||
``` | ||
|
||
This will deploy: | ||
- Kubernetes namespace, deployment, and service for the application | ||
|
||
```yaml | ||
apiVersion: v1 | ||
kind: Namespace | ||
metadata: | ||
name: webapi | ||
--- | ||
apiVersion: apps/v1 | ||
kind: Deployment | ||
metadata: | ||
name: webapi | ||
namespace: webapi | ||
spec: | ||
replicas: 3 | ||
selector: | ||
matchLabels: | ||
app: webapi | ||
template: | ||
metadata: | ||
labels: | ||
app: webapi | ||
spec: | ||
containers: | ||
- name: webapi | ||
image: ghcr.io/houssemdellai/containerapps-album-backend:v1 | ||
ports: | ||
- containerPort: 3500 | ||
--- | ||
apiVersion: v1 | ||
kind: Service | ||
metadata: | ||
name: webapi | ||
namespace: webapi | ||
spec: | ||
type: ClusterIP | ||
ports: | ||
- port: 80 | ||
targetPort: 3500 | ||
selector: | ||
app: webapi | ||
``` | ||
- Nginx Ingress Controller with an internal Load Balancer that uses static IP | ||
```yaml | ||
apiVersion: approuting.kubernetes.azure.com/v1alpha1 | ||
kind: NginxIngressController | ||
metadata: | ||
name: nginx-internal-static | ||
spec: | ||
ingressClassName: nginx-internal-static | ||
controllerNamePrefix: nginx-internal-static | ||
loadBalancerAnnotations: | ||
service.beta.kubernetes.io/azure-load-balancer-internal: "true" | ||
service.beta.kubernetes.io/azure-load-balancer-ipv4: "10.10.0.10" | ||
``` | ||
- Ingress resource to expose the application using the Ingress Controller | ||
```yaml | ||
apiVersion: networking.k8s.io/v1 | ||
kind: Ingress | ||
metadata: | ||
name: webapi | ||
namespace: webapi | ||
spec: | ||
ingressClassName: nginx-internal-static | ||
rules: | ||
- http: | ||
paths: | ||
- backend: | ||
service: | ||
name: webapi | ||
port: | ||
number: 80 | ||
path: / | ||
pathType: Prefix | ||
``` | ||
The application should now be exposed using the Ingress Controller. | ||
```sh | ||
kubectl run nginx --image=nginx | ||
# pod/nginx created | ||
root@nginx:/# curl 10.10.0.10/albums | ||
[{"id":1,"title":"You, Me and an App Id","artist":"Daprize","price":10.99,"image_url":"https://aka.ms/albums-daprlogo"},{"id":2,"title":"Seven Revision Army","artist":"The Blue-Green Stripes","price":13.99,"image_url":"https://aka.ms/albums-containerappslogo"},{"id":3,"title":"Scale It Up","artist":"KEDA Club","price":13.99,"image_url":"https://aka.ms/albums-kedalogo"},{"id":4,"title":"Lost in Translation","artist":"MegaDNS","price":12.99,"image_url":"https://aka.ms/albums-envoylogo"},{"id":5,"title":"Lock Down Your Love","artist":"V is for VNET","price":12.99,"image_url":"https://aka.ms/albums-vnetlogo"},{"id":6,"title":"Sweet Container O' Mine","artist":"Guns N Probeses","price":14.99,"image_url":"https://aka.ms/albums-containerappslogo"}] | ||
``` | ||
|
||
Now you can verify the application is exposed using the API Management instance. | ||
Get the Gateway URL, which should be something like `https://apim-external-aks-230-swc.azure-api.net`. | ||
Then paste the URL in the browser and add `/albums` to the end of the URL. You should see the same response as before. | ||
|
||
 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,49 @@ | ||
resource "azurerm_kubernetes_cluster" "aks" { | ||
name = "aks-cluster" | ||
location = azurerm_resource_group.rg.location | ||
resource_group_name = azurerm_resource_group.rg.name | ||
dns_prefix = "aks" | ||
kubernetes_version = "1.30.0" | ||
private_cluster_enabled = false | ||
|
||
network_profile { | ||
network_plugin = "azure" | ||
network_plugin_mode = "overlay" | ||
} | ||
|
||
default_node_pool { | ||
name = "mainpool" | ||
node_count = 3 | ||
vm_size = "Standard_B2als_v2" | ||
os_sku = "AzureLinux" | ||
vnet_subnet_id = azurerm_subnet.snet-aks.id | ||
enable_node_public_ip = false | ||
} | ||
|
||
identity { | ||
type = "SystemAssigned" | ||
} | ||
|
||
lifecycle { | ||
ignore_changes = [ | ||
default_node_pool.0.upgrade_settings | ||
] | ||
} | ||
} | ||
|
||
# Required to create internal Load Balancer for Nginx Ingress Controller | ||
resource "azurerm_role_assignment" "network-contributor" { | ||
scope = azurerm_subnet.snet-aks.id | ||
role_definition_name = "Network Contributor" | ||
principal_id = azurerm_kubernetes_cluster.aks.identity.0.principal_id | ||
} | ||
|
||
resource "terraform_data" "aks-get-credentials" { | ||
triggers_replace = [ | ||
azurerm_kubernetes_cluster.aks.id | ||
] | ||
|
||
provisioner "local-exec" { | ||
command = "az aks get-credentials -n ${azurerm_kubernetes_cluster.aks.name} -g ${azurerm_kubernetes_cluster.aks.resource_group_name} --overwrite-existing" | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,23 @@ | ||
resource "azurerm_api_management_api" "api-albums" { | ||
name = "api-albums" | ||
resource_group_name = azurerm_api_management.apim.resource_group_name | ||
api_management_name = azurerm_api_management.apim.name | ||
revision = "1" | ||
display_name = "api-albums" | ||
path = "albums" | ||
api_type = "http" # graphql, http, soap, and websocket | ||
protocols = ["http", "https"] | ||
service_url = "http://webapi.backend.svc.cluster.local/albums" # Service name of the webapi inside the AKS cluster | ||
subscription_required = false | ||
} | ||
|
||
resource "azurerm_api_management_api_operation" "operation-get-albums" { | ||
operation_id = "api-albums-get" | ||
api_name = azurerm_api_management_api.api-albums.name | ||
api_management_name = azurerm_api_management_api.api-albums.api_management_name | ||
resource_group_name = azurerm_api_management_api.api-albums.resource_group_name | ||
display_name = "GET" | ||
method = "GET" | ||
url_template = "/" | ||
description = "GET returns sample JSON file." | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,17 @@ | ||
resource "azurerm_api_management_gateway" "gateway" { | ||
name = "gateway-aks" | ||
api_management_id = azurerm_api_management.apim.id | ||
description = "API Management self-hosted gateway in an AKS cluster" | ||
|
||
location_data { | ||
name = "AKS on Azure" | ||
city = "example city" | ||
district = "example district" | ||
region = var.location | ||
} | ||
} | ||
|
||
resource "azurerm_api_management_gateway_api" "gateway-api" { | ||
gateway_id = azurerm_api_management_gateway.gateway.id | ||
api_id = azurerm_api_management_api.api-albums.id | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,27 @@ | ||
# resource "azurerm_public_ip" "pip-apim" { | ||
# name = "pip-apim" | ||
# location = azurerm_resource_group.rg.location | ||
# resource_group_name = azurerm_resource_group.rg.name | ||
# allocation_method = "Static" | ||
# sku = "Standard" | ||
# zones = ["1"] | ||
# domain_name_label = "apim-extern-${var.prefix}" | ||
# } | ||
|
||
resource "azurerm_api_management" "apim" { | ||
name = "apim-public-${var.prefix}" | ||
location = azurerm_resource_group.rg.location | ||
resource_group_name = azurerm_resource_group.rg.name | ||
publisher_name = "My Company" | ||
publisher_email = "houssem.dellai@live.com" | ||
sku_name = "Developer_1" | ||
virtual_network_type = "None" # External, Internal | ||
# public_ip_address_id = azurerm_public_ip.pip-apim.id | ||
public_network_access_enabled = true # false applies only when using private endpoint as the exclusive access method | ||
|
||
# virtual_network_configuration { | ||
# subnet_id = azurerm_subnet.snet-apim.id | ||
# } | ||
|
||
# depends_on = [azurerm_subnet_network_security_group_association.nsg-association] | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
resource "terraform_data" "deploy-app-to-aks" { | ||
triggers_replace = [ | ||
azurerm_kubernetes_cluster.aks.id | ||
] | ||
|
||
provisioner "local-exec" { | ||
command = "kubectl apply -f ../kubernetes/app.yaml" | ||
} | ||
|
||
depends_on = [ terraform_data.aks-get-credentials ] | ||
} |
Oops, something went wrong.