Skip to content

Commit

Permalink
update: Various Kyverno language fixes (#1143)
Browse files Browse the repository at this point in the history
  • Loading branch information
@niallthomson
niallthomson authored Oct 11, 2024
1 parent 0e51da7 commit 9dae1b4
Show file tree
Hide file tree
Showing 6 changed files with 82 additions and 91 deletions.
15 changes: 1 addition & 14 deletions manifests/modules/security/kyverno/.workshop/terraform/main.tf
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,17 +26,4 @@ module "kyverno_policies" {
]

depends_on = [module.kyverno]
}

module "policy_reporter" {
source = "aws-ia/eks-blueprints-addon/aws"
version = "1.1.1"

description = "Kyverno Policy Reporter which shows policy reports in a graphical web-based front end."
chart = "policy-reporter"
chart_version = "1.3.0"
namespace = "kyverno"
repository = "https://kyverno.github.io/policy-reporter/"

depends_on = [module.kyverno]
}
}
25 changes: 12 additions & 13 deletions website/docs/security/kyverno/baseline-pss.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,13 @@ title: "Enforcing Pod Security Standards"
sidebar_position: 72
---

As discussed in the introduction for [Pod Security Standards (PSS)](../pod-security-standards/) section, there are 3 pre-defined Policy levels, **Privileged**, **Baseline**, and **Restricted**. While it is recommended to setup a Restricted PSS, it can cause unintended behavior on the application level unless properly set. To get started it is recommended to setup a Baseline Policy that will prevent known Privileged escalations such as Containers accessing HostProcess, HostPath, HostPorts or allow traffic snooping for example, being possible to setup individual policies to restrict or disallow those privileged access to containers.
As discussed in the introduction for [Pod Security Standards (PSS)](../pod-security-standards/) section, there are three pre-defined policy levels: **Privileged**, **Baseline**, and **Restricted**. While implementing a Restricted PSS is recommended, it can cause unintended behavior at the application level unless properly configured. To get started, it's recommended to set up a Baseline Policy that will prevent known privileged escalations such as containers accessing HostProcess, HostPath, HostPorts, or allowing traffic snooping. Individual policies can then be set up to restrict or disallow these privileged accesses to containers.

A Kyverno Baseline Policy will help to restrict all the known privileged escalation under a single policy, and also maintain and update the Policy regularly adding the latest found vulnerabilities to the Policy.
A Kyverno Baseline Policy helps restrict all known privileged escalations under a single policy. It also allows for regular maintenance and updates to incorporate the latest discovered vulnerabilities into the policy.

Privileged containers can do almost everything that the host can do and are often used in CI/CD pipelines to allow building and publishing Container images.
With the now fixed [CVE-2022-23648](https://github.com/containerd/containerd/security/advisories/GHSA-crp2-qrr5-8pq7) any bad actor, could escape the privileged container by abusing the Control Groups `release_agent` functionality to execute arbitrary commands on the container host.
Privileged containers can perform almost all actions that the host can do and are often used in CI/CD pipelines to allow building and publishing container images. With the now fixed [CVE-2022-23648](https://github.com/containerd/containerd/security/advisories/GHSA-crp2-qrr5-8pq7), a malicious actor could escape the privileged container by exploiting the Control Groups `release_agent` functionality to execute arbitrary commands on the container host.

In this lab we will run a privileged Pod on our EKS cluster. To do that, execute the following command:
In this lab, we will run a privileged Pod on our EKS cluster. Execute the following command:

```bash
$ kubectl run privileged-pod --image=nginx --restart=Never --privileged
Expand All @@ -19,24 +18,24 @@ $ kubectl delete pod privileged-pod
pod "privileged-pod" deleted
```

In order to avoid such escalated privileged capabilities and avoid unauthorized use of above permissions, it's recommended to setup a Baseline Policy using Kyverno.
To prevent such escalated privileged capabilities and avoid unauthorized use of these permissions, it's recommended to set up a Baseline Policy using Kyverno.

The baseline profile of the Pod Security Standards is a collection of the most basic and important steps that can be taken to secure Pods. Beginning with Kyverno 1.8, an entire profile may be assigned to the cluster through a single rule. To check more on the privileges blocked by Baseline Profile, please refer [here](https://kyverno.io/policies/#:~:text=Baseline%20Pod%20Security%20Standards,cluster%20through%20a%20single%20rule)
The baseline profile of the Pod Security Standards is a collection of the most fundamental and crucial steps that can be taken to secure Pods. Starting from Kyverno 1.8, an entire profile can be assigned to the cluster through a single rule. To learn more about the privileges blocked by the Baseline Profile, please refer to the [Kyverno documentation](https://kyverno.io/policies/#:~:text=Baseline%20Pod%20Security%20Standards,cluster%20through%20a%20single%20rule).

```file
manifests/modules/security/kyverno/baseline-policy/baseline-policy.yaml
```

Notice that he above policy is in `Enforce` mode, and will block any requests to create privileged Pod.
Note that the above policy is in `Enforce` mode and will block any requests to create privileged Pods.

Go ahead and apply the Baseline Policy.
Go ahead and apply the Baseline Policy:

```bash
$ kubectl apply -f ~/environment/eks-workshop/modules/security/kyverno/baseline-policy/baseline-policy.yaml
clusterpolicy.kyverno.io/baseline-policy created
```

Now, try to run the privileged Pod again.
Now, try to run the privileged Pod again:

```bash expectError=true
$ kubectl run privileged-pod --image=nginx --restart=Never --privileged
Expand All @@ -49,10 +48,10 @@ baseline-policy:
Validation rule 'baseline' failed. It violates PodSecurity "baseline:latest": ({Allowed:false ForbiddenReason:privileged ForbiddenDetail:container "privileged-pod" must not set securityContext.privileged=true})
```

As seen the creation failed, because it isn't in compliance with our Baseline Policy set on the Cluster.
As you can see, the creation failed because it doesn't comply with our Baseline Policy set on the cluster.

### Note on Auto-Generated Policies

PSA operates at the Pod level, but in practice Pods are usually managed by Pod controllers, like Deployments. Having no indication of Pod security errors at the Pod controller level can make issues complex to troubleshoot. The PSA enforce mode is the only PSA mode that stops Pods from being created, however PSA enforcement doesn’t act at the Pod controller level. To improve this experience, it's recommended that PSA `warn` and `audit` modes are also used with `enforce`. With that PSA will indicate that the controller resources are trying to create Pods that would fail with the applied PSS level.
Pod Security Admission (PSA) operates at the Pod level, but in practice, Pods are usually managed by Pod controllers like Deployments. Having no indication of Pod security errors at the Pod controller level can make issues complex to troubleshoot. The PSA enforce mode is the only PSA mode that prevents Pods from being created; however, PSA enforcement doesn't act at the Pod controller level. To improve this experience, it's recommended that PSA `warn` and `audit` modes are also used with `enforce`. This way, PSA will indicate that the controller resources are trying to create Pods that would fail with the applied PSS level.

Using PaC solutions with Kubernetes presents another challenge of writing and maintaining policies to cover all the different resources used within clusters. With the [Kyverno Auto-Gen Rules for Pod Controllers](https://kyverno.io/docs/writing-policies/autogen/) feature, the Pod policies auto-generate associated Pod controller (Deployment, DaemonSet, etc.) policies. This Kyverno feature increases the expressive nature of policies and reduces the effort to maintain policies for associated resources. improving PSA user experience where controllers resources are not prevented from progressing, while the underlying Pods are.
Using Policy-as-Code (PaC) solutions with Kubernetes presents another challenge of writing and maintaining policies to cover all the different resources used within clusters. With the [Kyverno Auto-Gen Rules for Pod Controllers](https://kyverno.io/docs/writing-policies/autogen/) feature, the Pod policies auto-generate associated Pod controller (Deployment, DaemonSet, etc.) policies. This Kyverno feature enhances the expressive nature of policies and reduces the effort to maintain policies for associated resources, improving the PSA user experience where controller resources are not prevented from progressing while the underlying Pods are.
48 changes: 24 additions & 24 deletions website/docs/security/kyverno/creating-policy.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,44 +3,44 @@ title: "Creating a Simple Policy"
sidebar_position: 71
---

To get an understanding of Kyverno Policies, we will start our lab with a simple Pod Label requirement. As you may know, Labels in Kubernetes can be used to tag objects and resources in the Cluster.
To gain an understanding of Kyverno Policies, we'll start our lab with a simple Pod Label requirement. As you may know, Labels in Kubernetes are used to tag objects and resources in the cluster.

Below we have a sample policy requiring a Label `CostCenter`.
Below is a sample policy requiring a Label `CostCenter`:

```file
manifests/modules/security/kyverno/simple-policy/require-labels-policy.yaml
```

Kyverno has 2 kinds of Policy resources, **ClusterPolicy** used for Cluster-Wide Resources and **Policy** used for Namespaced Resources. The example above shows a ClusterPolicy. Take sometime to dive deep and check the below details in the configuration.
Kyverno has two kinds of Policy resources: **ClusterPolicy** used for Cluster-Wide Resources and **Policy** used for Namespaced Resources. The example above shows a ClusterPolicy. Take some time to examine the following details in the configuration:

- Under the spec section of the Policy, there is a an attribute `validationFailureAction` it tells Kyverno if the resource being validated should be allowed but reported `Audit` or blocked `Enforce`. Defaults to Audit, the example is set to Enforce.
- The `rules` is one or more rules to be validated.
- The `match` statement sets the scope of what will be checked. In this case, it is any `Pod` resource.
- The `validate` statement tries to positively check what is defined. If the statement, when compared with the requested resource, is true, it is allowed. If false, it is blocked.
- Under the `spec` section of the Policy, there's an attribute `validationFailureAction`. It tells Kyverno if the resource being validated should be allowed but reported (`Audit`) or blocked (`Enforce`). The default is `Audit`, but our example is set to `Enforce`.
- The `rules` section contains one or more rules to be validated.
- The `match` statement sets the scope of what will be checked. In this case, it's any `Pod` resource.
- The `validate` statement attempts to positively check what is defined. If the statement, when compared with the requested resource, is true, it's allowed. If false, it's blocked.
- The `message` is what gets displayed to a user if this rule fails validation.
- The `pattern` object defines what pattern will be checked in the resource. In this case, it is looking for `metadata.labels` with `CostCenter`.
- The `pattern` object defines what pattern will be checked in the resource. In this case, it's looking for `metadata.labels` with `CostCenter`.

The Above Example Policy, will block any Pod Creation which doesn't have the label `CostCenter`.
This example Policy will block any Pod creation that doesn't have the label `CostCenter`.

Create the policy using the following command.
Create the policy using the following command:

```bash
$ kubectl apply -f ~/environment/eks-workshop/modules/security/kyverno/simple-policy/require-labels-policy.yaml

clusterpolicy.kyverno.io/require-labels created
```

Next, take a look on the Pods running in the `ui` Namespace, notice the applied labels.
Next, take a look at the Pods running in the `ui` Namespace and notice the applied labels:

```bash
$ kubectl -n ui get pods --show-labels
NAME READY STATUS RESTARTS AGE LABELS
ui-67d8cf77cf-d4j47 1/1 Running 0 9m app.kubernetes.io/component=service,app.kubernetes.io/created-by=eks-workshop,app.kubernetes.io/instance=ui,app.kubernetes.io/name=ui,pod-template-hash=67d8cf77cf
```

Check the running Pod doesn't have the required Label and Kyverno didn't terminate it, this happened because as seen earlier, Kyverno operates as an `AdmissionController` and will not interfere in resources that already exist in the cluster.
Notice that the running Pod doesn't have the required Label, and Kyverno didn't terminate it. This is because Kyverno operates as an `AdmissionController` and won't interfere with resources that already exist in the cluster.

However if you delete the running Pod, it won't be able to be recreated since it doesn't have the required Label. Go ahead and delete de Pod running in the `ui` Namespace.
However, if you delete the running Pod, it won't be able to be recreated since it doesn't have the required Label. Go ahead and delete the Pod running in the `ui` Namespace:

```bash
$ kubectl -n ui delete pod --all
Expand All @@ -49,7 +49,7 @@ $ kubectl -n ui get pods
No resources found in ui namespace.
```

As mentioned, the Pod was not recreated, try to force a rollout of the `ui` deployment.
As mentioned, the Pod was not recreated. Try to force a rollout of the `ui` deployment:

```bash expectError=true
$ kubectl -n ui rollout restart deployment/ui
Expand All @@ -64,7 +64,7 @@ require-labels:

The rollout failed with the admission webhook denying the request due to the `require-labels` Kyverno Policy.

You can also check this `error` message describing the `ui` deployment, or visualizing the `events` in the `ui` Namespace.
You can also check this `error` message by describing the `ui` deployment or viewing the `events` in the `ui` Namespace:

```bash
$ kubectl -n ui describe deployment ui
Expand All @@ -80,7 +80,7 @@ $ kubectl -n ui get events | grep PolicyViolation
9m Warning PolicyViolation deployment/ui policy require-labels/autogen-check-team fail: validation error: Label 'CostCenter' is required to deploy the Pod. rule autogen-check-team failed at path /spec/template/metadata/labels/CostCenter/
```

Now add the required label `CostCenter` to the `ui` Deployment, using the Kustomization patch below.
Now add the required label `CostCenter` to the `ui` Deployment, using the Kustomization patch below:

```kustomization
modules/security/kyverno/simple-policy/ui-labeled/deployment.yaml
Expand All @@ -100,29 +100,29 @@ NAME READY STATUS RESTARTS AGE LABELS
ui-5498685db8-k57nk 1/1 Running 0 60s CostCenter=IT,app.kubernetes.io/component=service,app.kubernetes.io/created-by=eks-workshop,app.kubernetes.io/instance=ui,app.kubernetes.io/name=ui,pod-template-hash=5498685db8
```

As you can see the admission webhook successfully validated the Policy and the Pod was created with the correct Label `CostCenter=IT`!
As you can see, the admission webhook successfully validated the Policy and the Pod was created with the correct Label `CostCenter=IT`!

### Mutating Rules

In the above examples, you checked how Validation Policies work in their default behavior defined in `validationFailureAction`. However Kyverno can also be used to manage Mutating rules within the Policy, in order to modify any API Requests to satisfy or enforce the specified requirements on the Kubernetes resources. The resource mutation occurs before validation, so the validation rules will not contradict the changes performed by the mutation section.
In the above examples, you checked how Validation Policies work in their default behavior defined in `validationFailureAction`. However, Kyverno can also be used to manage Mutating rules within the Policy, to modify any API Requests to satisfy or enforce the specified requirements on the Kubernetes resources. The resource mutation occurs before validation, so the validation rules will not contradict the changes performed by the mutation section.

Below is a sample Policy with a mutation rule defined, which will be used to automatically add our label `CostCenter=IT` as default to any `Pod`.
Below is a sample Policy with a mutation rule defined, which will be used to automatically add our label `CostCenter=IT` as default to any `Pod`:

```file
manifests/modules/security/kyverno/simple-policy/add-labels-mutation-policy.yaml
```

Notice the `mutate` section, under the ClusterPolicy `spec`.
Notice the `mutate` section under the ClusterPolicy `spec`.

Go ahead, and create the above Policy using the following command.
Go ahead and create the above Policy using the following command:

```bash
$ kubectl apply -f ~/environment/eks-workshop/modules/security/kyverno/simple-policy/add-labels-mutation-policy.yaml

clusterpolicy.kyverno.io/add-labels created
```

In order to validate the Mutation Webhook, lets this time rollout the `assets` Deployment without explicitly adding a label:
To validate the Mutation Webhook, let's roll out the `assets` Deployment without explicitly adding a label:

```bash
$ kubectl -n assets rollout restart deployment/assets
Expand All @@ -131,7 +131,7 @@ $ kubectl -n assets rollout status deployment/assets
deployment "assets" successfully rolled out
```

Validate the automatically added label `CostCenter=IT` to the Pod to meet the policy requirements, resulting a successful Pod creation even with the Deployment not having the label specified:
Validate that the label `CostCenter=IT` was automatically added to the Pod to meet the policy requirements, resulting in a successful Pod creation even though the Deployment didn't have the label specified:

```bash
$ kubectl -n assets get pods --show-labels
Expand All @@ -141,4 +141,4 @@ assets-bb88b4789-kmk62 1/1 Running 0 25s CostCenter=IT,app.ku

It's also possible to mutate existing resources in your Amazon EKS Clusters with Kyverno Policies using `patchStrategicMerge` and `patchesJson6902` parameters in your Kyverno Policy.

This was just a simple example of labels for our Pods with Validating and Mutating rules. This can be applied to various scenarios such as restricting Images from unknown registries, adding Data to Config Maps, Tolerations and much more. In the next upcoming labs, you will go through some more advanced use-cases.
This was just a simple example of labels for our Pods with Validating and Mutating rules. This can be applied to various scenarios such as restricting images from unknown registries, adding data to ConfigMaps, setting tolerations, and much more. In the upcoming labs, you will explore some more advanced use-cases.
Loading

0 comments on commit 9dae1b4

Please sign in to comment.