Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: namespace-defaults - adding sa for wi and update network policy #518

Merged
merged 1 commit into from
Sep 8, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
91 changes: 81 additions & 10 deletions solutions/gke/kubernetes/namespace-defaults/README.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,89 @@
# Namepsace Defaults
# Namespace Defaults

Landing zone v2 subpackage.
Depends on package `cluster-defaults`.
This package deploys to a GKE cluster.
It should be deployed once per workload namespace.

This package deploys a workload namespace and it's associated configuration.
## Description

---
Resources list:
This solution will create a dedicated namespace intended to contain an applications. In addition to the namespace, it also creates a number of related namespaced Kubernetes resources that should be created and configured by default.

- namespace
- resourcequota
- limitrange
- networkpolicies
- rolebinding for an application team
- a choice of Continuous Delivery(CD) solution which offer ConfigSync (Default) or ServiceAccount permission for external tools.
Below table describes components provisioned by this solution.

| **Component** | **Filename** |*Description** |
|------------------------------------|----------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------|
| Limit Range | limitrange.yaml | Implements a `LimitRange` to place constraints on the resource allocations by application pods within the namespace. |
| Service Account | namespace-sa.yaml | Implements a workload identity service account that can be used by pods to access GCP API |
| Namespace | namespace.yaml | Implements the namespace. |
| Network Policy | networkpolicy.yaml | Implements a default network policy to enforce network security within the namespace. |
| Resource Quotas | resourcequota.yaml | Implements a `ResourceQuota` constraint that limits aggregate resource consumption in the namespace. |
| Rolebinding for Gateway HTTPRoutes | rolebinding-httproute-admin.yaml | Implements permissions required by the Gateway controller to manage HTTPRoute resources. |
| Rolebinding for Client Team | rolebinding-human-view.yaml | Implements read/view permissions to be granted to the application team. |
| Config Sync | cd/gitops-config-sync.yaml | Implements a GitOps solution using ConfigSync to deploy/update of Kubernetes resources related to application. |
| Rolebinding for CI/CD | cd/cd-rolebinding.yaml | Implements permissions to enable an external CI/CD system to deploy/update of Kubernetes resources related to application. **NOTE**: This component is not enabled by default. |

### Limit Range

By default, containers run with unbounded compute resources on a Kubernetes cluster. Kubernetes resource quotas can restrict consumption and creation of cluster resources (such as CPU time, memory, and persistent storage) within a specified namespace. Within a namespace, a Pod can consume as much CPU and memory as is allowed by the `ResourceQuotas` that apply to that namespace. You might also be concerned about making sure that a single object cannot monopolize all available resources within a namespace.

A `LimitRange` is a constraint on the resource allocations for application Pods in a namespace. They are useful so that a single object cannot monopolize all available resources within a namespace.

A `LimitRange` provides constraints that can:

- Enforce minimum and maximum compute resources usage per Pod or Container in a namespace.
- Enforce minimum and maximum storage request per PersistentVolumeClaim in a namespace.
- Enforce a ratio between request and limit for a resource in a namespace.
- Set default request/limit for compute resources in a namespace and automatically inject them to Containers at runtime.

The values should be configured based on requirements of the applications deployed in the namespace and size of nodes in the cluster.

### Network Policy

Network policy specify how a pod is allowed to communicate over the network and control the traffic flow at the IP address or port level (OSI layer 3 or 4). They serve as a Kubernetes level implementation of firewall rules.

The implementation sets a number of default rules for pods within the namespace:

- Allow ingress/egress communication to all pods within the same namespace.
- Allow ingress from gateway namespace.
- Allow ingress from Google Network Load Balancer for health checks
- Allow egress to Google metadata server
- Allow egress for Google Workload Identity
- Allow egress for to Google DNS services
- Allow egress to Google API.

### Resource Quota

A Kubernetes `ResourceQuota` API provides constraints that limit aggregate resource consumption per namespace. It can limit the quantity of objects that can be created in a namespace by type, as well as the total amount of compute resources that may be consumed by resources in that namespace.
If creating or updating a resource in a namespace violates a quota constraint, the request will fail (HTTP status code 403 FORBIDDEN) with a message explaining the constraint that would have been violated.

Some common valued that are set in resource quotas are:

- The memory request total for all Pods in a namespace
- The memory limit total for all Pods in a namespace
- The CPU request total for all Pods in a namespace
- The CPU limit total for all Pods in a namespace
- Total number of Pods that can run in a namespace

### Rolebinding for Gateway HTTPRoutes

Gateway API is Kubernetes API that implements ingress of traffic into a Kubernetes cluster. The Gateway Controller is the Kubernetes component that is responsible for this task. It requires permissions to manage Kubernetes `HTTPRoute` resource in order to achieve this functionality.

### Rolebinding for Client Team

Application team should be granted read/view permissions to the namespace. This can be achieved using the standard `view` Kubernetes cluster role.

To efficiently manage tenant permissions in a cluster, RBAC permissions should be bound to Google Groups. The membership of those groups are maintained by Google Workspace administrators.

### Config Sync

Implements a GitOps solution that observes a git repository and applies the manifest located in the specified folder.

### Rolebinding for CI/CD

> **NOTE**: This component is not enabled by default.

Application teams may want to use a third party CI/CD solution for deploying application inside their Kubernetes namespace. To facilitate this scenario, a service account for the deployment system is needed. This service account should be granted sufficient permissions to the namespace to deploy/update Kubernetes resources related to their application.
This can be achieved using the standard `edit` Kubernetes cluster role.

Kubernetes RBAC permissions should be bound to Google IAM Service Account representing the CI/CD system.
23 changes: 23 additions & 0 deletions solutions/gke/kubernetes/namespace-defaults/namespace-sa.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Copyright 2021 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#########
# Creates Kuberenetes Service Account to be used by applications deployed in the namespace
# Kubernetes Service Account is annotated with the Google Service Account to which it is bound using workload identity.
apiVersion: v1
kind: ServiceAccount
metadata:
annotations:
iam.gke.io/gcp-service-account: workload-name-sa@project-id.iam.gserviceaccount.com # kpt-set: ${workload-name}-sa@${project-id}.iam.gserviceaccount.com
name: workload-name-sa # kpt-set: ${workload-name}-sa
namespace: workload-name # kpt-set: ${workload-name}
24 changes: 24 additions & 0 deletions solutions/gke/kubernetes/namespace-defaults/networkpolicy.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,7 @@ spec:
# Network policy and Workload Identity
# For clusters running GKE version 1.21.0-gke.1000 and later, allow egress to 169.254.169.252/32 on port 988
# For clusters running GKE Dataplane V2, allow egress to 169.254.169.254/32 on port 80
# Allow access to NodeLocal DNSCache on ip 169.254.20.10 and port 53
- to:
- ipBlock:
cidr: 169.254.169.252/32
Expand All @@ -102,3 +103,26 @@ spec:
- port: 53
protocol: TCP
- port: 80
- to:
- ipBlock:
cidr: 169.254.20.10/32
ports:
- port: 53
protocol: UDP
- port: 53
protocol: TCP
---
# Allow egress for GCP API
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
namespace: workload-name # kpt-set: ${workload-name}
name: allow-egrees-to-gcp-api
spec:
podSelector: {}
egress:
- to:
- ipBlock:
cidr: 10.255.255.254/32
ports:
- port: 443
8 changes: 8 additions & 0 deletions solutions/gke/kubernetes/namespace-defaults/setters.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,14 @@ data:
##########################
#
# Follow instructions specific to each section.
#
##########################
# Project
##########################
#
# project-id created by the client-project-setup package
project-id: project-12345
#
##########################
# Client
##########################
Expand Down