From e1599951a2e01d1b28e327a67d022a7e2a43fc91 Mon Sep 17 00:00:00 2001 From: Abhishek Raut Date: Wed, 9 Sep 2020 13:07:20 -0700 Subject: [PATCH] Add Antrea NetworkPolicy documentation (#1194) Add documentation regarding the namespaced Antrea NetworkPolicy CRD --- docs/network-policy.md | 246 ++++++++++++++++++++++++++++------------- 1 file changed, 168 insertions(+), 78 deletions(-) diff --git a/docs/network-policy.md b/docs/network-policy.md index f456996b291..32416b0dfa3 100644 --- a/docs/network-policy.md +++ b/docs/network-policy.md @@ -1,4 +1,4 @@ -# Antrea Network Policies +# Antrea Policy CRDs ## Table of Contents @@ -8,30 +8,37 @@ - [Tier CRDs](#tier-crds) - [ClusterNetworkPolicy](#clusternetworkpolicy) - [The ClusterNetworkPolicy resource](#the-clusternetworkpolicy-resource) - - [Rule enforcement based on priorities](#rule-enforcement-based-on-priorities) - [Behavior of to and from selectors](#behavior-of-to-and-from-selectors) - [Key differences from K8s NetworkPolicy](#key-differences-from-k8s-networkpolicy) +- [Antrea NetworkPolicy](#antrea-networkpolicy) + - [The Antrea NetworkPolicy resource](#the-antrea-networkpolicy-resource) + - [Key differences from ClusterNetworkPolicy](#key-differences-from-clusternetworkpolicy) +- [Antrea Policy ordering based on priorities](#antrea-policy-ordering-based-on-priorities) + - [Ordering based on Tier priority](#ordering-based-on-tier-priority) + - [Ordering based on policy priority](#ordering-based-on-policy-priority) + - [Rule enforcement based on priorities](#rule-enforcement-based-on-priorities) - [Notes](#notes) ## Summary -Antrea supports standard K8s NetworkPolicies to secure traffic between Pods. These -NetworkPolicies are written from an application developer's perspective, hence they -lack the ability to gain a finer-grained control over the security policies that -a cluster administrator would require. This document describes a few new CRDs -supported by Antrea to provide the administrator with more control over -security within the cluster, and which are meant to co-exist with and complement the -K8s NetworkPolicy. +Antrea supports standard K8s NetworkPolicies to secure traffic between Pods. +These NetworkPolicies are written from an application developer's perspective, +hence they lack the ability to gain a finer-grained control over the security +policies that a cluster administrator would require. This document describes a +few new CRDs supported by Antrea to provide the administrator with more control +over security within the cluster, and which are meant to co-exist with and +complement the K8s NetworkPolicy. ## Tier -Antrea supports grouping Antrea NetworkPolicies together in a tiered fashion -to provide a hierarchy of security policies. This is achieved by setting the `tier` -field when defining Antrea NetworkPolicies (e.g. a ClusterNetworkPolicy object) to -the appropriate tier name. Each tier has a priority associated with it, which -determines its relative order among all tiers. +Antrea supports grouping Antrea Policy CRDs together in a tiered fashion +to provide a hierarchy of security policies. This is achieved by setting the +`tier` field when defining an Antrea Policy CRD (e.g. a ClusterNetworkPolicy +object) to the appropriate tier name. Each tier has a priority associated with +it, which determines its relative order among all tiers. -**Note**: K8s NetworkPolicies will be enforced once all tiers have been enforced. +**Note**: K8s NetworkPolicies will be enforced once all tiers have been +enforced. ### Static tiers @@ -42,15 +49,16 @@ relative order of precedence: Emergency > SecurityOps > NetworkOps > Platform > Application ``` -Thus, all ClusterNetworkPolicies associated with "Emergency" tier will be -enforced before any other ClusterNetworkPolicy associated with any other tier, -until a match occurs, in which case the policy rule's `action` will be -applied. The "Application" tier carries the lowest precedence, and any -ClusterNetworkPolicy without a `tier` name set in its spec will be -associated with the "Application" tier. Even though the policies associated -with the "Application" tier carry the lowest precedence amongst all the tiers, -they are still enforced before K8s NetworkPolicies. Thus, admin created tiered -policies have a higher precedence over developer-created K8s NetworkPolicies. +Thus, all Antrea Policy resources associated with "Emergency" tier will be +enforced before any other Antrea Policy resource associated with any other +tier, until a match occurs, in which case the policy rule's `action` will be +applied. The "Application" tier carries the lowest precedence, and any Antrea +Policy resource without a `tier` name set in its spec will be associated with +the "Application" tier. Even though the policies associated with the +"Application" tier carry the lowest precedence amongst all the tiers, they are +still enforced before K8s NetworkPolicies. Thus, admin-created tiered Antrea +Policy CRDs have a higher precedence than developer-created K8s +NetworkPolicies. ### Tier CRDs @@ -62,12 +70,12 @@ will be available soon and will replace the existing static tiers. ## ClusterNetworkPolicy -ClusterNetworkPolicy is a specification of how workloads within a cluster -communicate with each other and other external endpoints. -The ClusterNetworkPolicy is supposed to aid cluster admins to configure -the security policy for the cluster, unlike K8s NetworkPolicy, which is -aimed towards developers to secure their apps and affects Pods within the -Namespace in which the K8s NetworkPolicy is created. +ClusterNetworkPolicy, one of the two Antrea Policy CRDs introduced, is a +specification of how workloads within a cluster communicate with each other and +other external endpoints. The ClusterNetworkPolicy is supposed to aid cluster +admins to configure the security policy for the cluster, unlike K8s +NetworkPolicy, which is aimed towards developers to secure their apps and +affects Pods within the Namespace in which the K8s NetworkPolicy is created. Rules belonging to ClusterNetworkPolicies are enforced before any rule belonging to a K8s NetworkPolicy. @@ -94,7 +102,7 @@ ConfigMap as follows: ### The ClusterNetworkPolicy resource An example ClusterNetworkPolicy might look like this: -``` +```yaml apiVersion: security.antrea.tanzu.vmware.com/v1alpha1 kind: ClusterNetworkPolicy metadata: @@ -148,10 +156,10 @@ In the example, the policy applies to Pods, which either match the labels "role=db" in all the Namespaces, or are from Namespaces which match the labels "env=prod". -**priority**: The `priority` field determines the relative priority of the policy -among all ClusterNetworkPolicies in the given cluster. This field is mandatory. -A lower priority value indicates higher precedence. Priority values can range -from 1.0 to 10000.0. +**priority**: The `priority` field determines the relative priority of the +policy among all ClusterNetworkPolicies in the given cluster. This field is +mandatory. A lower priority value indicates higher precedence. Priority values +can range from 1.0 to 10000.0. **Note**: Policies with the same priorities will be enforced indeterministically. Users should therefore take care to use priorities to ensure the behavior they expect. @@ -168,41 +176,16 @@ The example policy contains a single rule, which allows matched traffic on a single port, from one of two sources: the first specified by a `podSelector` and the second specified by a combination of a `podSelector` and a `namespaceSelector`. -**Note**: The order in which the ingress rules are set matter, i.e. rules will be -enforced in the order in which they are written. +**Note**: The order in which the ingress rules are set matter, i.e. rules will +be enforced in the order in which they are written. -**egress**: Each ClusterNetworkPolicy may consist of zero or more ordered set of -egress rules. Each rule, depending on the `action` field of the rule, allows +**egress**: Each ClusterNetworkPolicy may consist of zero or more ordered set +of egress rules. Each rule, depending on the `action` field of the rule, allows or drops traffic which matches both the `to` and `ports` sections. The example policy contains a single rule, which drops matched traffic on a single port, to the 10.0.10.0/24 subnet specified by the `ipBlock` field. -**Note**: The order in which the egress rules are set matter, i.e. rules will be -enforced in the order in which they are written. - -### Rule enforcement based on priorities - -With the introduction of tiers, ClusterNetworkPolicies are first enforced -based on the tier to which they are associated with. i.e. all CNP belonging -to the "Emergency" tier are enforced first, followed by policies associated with -the "SecurityOps" tier and so on, until the "Application" tier policies are -enforced. Within a tier, rules belonging to Cluster NetworkPolicy CRDs are -associated with various priorities, such as the `priority` at the CNP level and -the priority at the rule level. Overall, the Cluster Policy with the highest -precedence (lowest priority number value) is enforced first. Within this policy, -rules are enforced in the order in which they are set. For example, consider the -following: - -- CNP1{tier: Application, priority: 10, ingressRules: [ir1.1, ir1.2], egressRules: [er1.1, er1.2]} -- CNP2{tier: Application, priority: 15, ingressRules: [ir2.1, ir2.2], egressRules: [er2.1, er2.2]} -- CNP3{tier: Emergency, priority: 20, ingressRules: [ir3.1, ir3.2], egressRules: [er3.1, er3.2]} - -This translates to the following order: -- Ingress rules: ir3.1 > ir3.2 > ir1.1 -> ir1.2 -> ir2.1 -> ir2.2 -- Egress rules: er3.1 > er3.2 > er1.1 -> er1.2 -> er2.1 -> er2.2 - -Once a rule is matched, it is executed based on the action set. If none of the -CNP rules match, the packet is then enforced for rules created for K8s NP. -Hence, CNP take precedence over K8s NP. +**Note**: The order in which the egress rules are set matter, i.e. rules will +be enforced in the order in which they are written. ### Behavior of `to` and `from` selectors @@ -212,21 +195,21 @@ section or egress `to` section: **podSelector**: This selects particular Pods from all Namespaces as "sources", if set in `ingress` section, or as "destinations", if set in `egress` section. -**namespaceSelector**: This selects particular Namespaces for which all Pods are -grouped as `ingress` "sources" or `egress` "destinations". +**namespaceSelector**: This selects particular Namespaces for which all Pods +are grouped as `ingress` "sources" or `egress` "destinations". -**podSelector** and **namespaceSelector**: A single to/from entry that specifies -both namespaceSelector and podSelector selects particular Pods within +**podSelector** and **namespaceSelector**: A single to/from entry that +specifies both namespaceSelector and podSelector selects particular Pods within particular Namespaces. -**ipBlock**: This selects particular IP CIDR ranges to allow as `ingress` "sources" -or `egress` "destinations". These should be cluster-external IPs, since Pod IPs are -ephemeral and unpredictable. +**ipBlock**: This selects particular IP CIDR ranges to allow as `ingress` +"sources" or `egress` "destinations". These should be cluster-external IPs, +since Pod IPs are ephemeral and unpredictable. ### Key differences from K8s NetworkPolicy -- ClusterNetworkPolicy is at the cluster scope, hence a `podSelector` without any - `namespaceSelector` selects Pods from all Namespaces. +- ClusterNetworkPolicy is at the cluster scope, hence a `podSelector` without + any `namespaceSelector` selects Pods from all Namespaces. - There is no automatic isolation of Pods on being selected in appliedTo. - Ingress/Egress rules in ClusterNetworkPolicy has an `action` field which specifies whether the matched rule allows or drops the traffic. @@ -236,8 +219,115 @@ ephemeral and unpredictable. - Rules assume the priority in which they are written. i.e. rule set at top takes precedence over a rule set below it. +## Antrea NetworkPolicy + +Antrea NetworkPolicy is another Policy CRD, which is similar to the +ClusterNetworkPolicy CRD, however its scope is limited to a Namespace. +The purpose of introducing this CRD is to allow admins to take advantage of +advanced NetworkPolicy features and apply them within a Namespace to +complement the K8s NetworkPolicies. Similar to the ClusterNetworkPolicy +resource, Antrea NetworkPolicy can also be associated with Tiers. + +**Note**: Antrea NetworkPolicy is currently in "Alpha" stage and is enabled +along with Tiers and ClusterNetworkPolicy as part of the `AntreaPolicy` +feature gate. + +### The Antrea NetworkPolicy resource + +An example Antrea NetworkPolicy might look like this: +```yaml +apiVersion: security.antrea.tanzu.vmware.com/v1alpha1 +kind: NetworkPolicy +metadata: + name: test-anp + namespace: default +spec: + priority: 5 + tier: SecurityOps + appliedTo: + - podSelector: + matchLabels: + role: db + ingress: + - action: Allow + from: + - podSelector: + matchLabels: + role: frontend + - podSelector: + matchLabels: + role: nondb + namespaceSelector: + matchLabels: + role: db + ports: + - protocol: TCP + port: 8080 + egress: + - action: Drop + to: + - ipBlock: + cidr: 10.0.10.0/24 + ports: + - protocol: TCP + port: 5978 +``` + +### Key differences from ClusterNetworkPolicy + +Antrea NetworkPolicy shares it's spec with ClusterNetworkPolicy. However, +the following documents some of the key differences between the two Antrea +Policy CRDs. + +- Antrea NetworkPolicy is Namespaced while ClusterNetworkPolicy operates at + cluster scope. +- Unlike the `appliedTo` in a ClusterNetworkPolicy, setting a + `namespaceSelector` in the `appliedTo` field is forbidden. +- `podSelector` without a `namespaceSelector`, set within a NetworkPolicy Peer + of any rule, selects Pods from the Namespace in which the Antrea + NetworkPolicy is created. This behavior is similar to the K8s NetworkPolicy. + +## Antrea Policy ordering based on priorities + +Antrea Policy CRDs are ordered based on priorities set at various levels. + +### Ordering based on Tier priority + +With the introduction of tiers, Antrea Policies, like ClusterNetworkPolicies, +are first enforced based on the tier to which they are associated. i.e. all +policies belonging to the "Emergency" tier are enforced first, followed by +policies belonging to the "SecurityOps" tier and so on, until the "Application" +tier policies are enforced. + +### Ordering based on policy priority + +Within a tier, Antrea Policy CRDs are ordered by the `priority` at the policy +level. Thus, the policy with the highest precedence (lowest priority number +value) is enforced first. This ordering is performed solely based on the +`priority` assigned as opposed to the "Kind" of the resource, i.e. the relative +ordering between a [ClusterNetworkPolicy resource](#clusternetworkpolicy) and an [Antrea NetworkPolicy +resource](#antrea-networkpolicy) within a Tier depends only on the `priority` +set in each of the two resources. + +### Rule enforcement based on priorities + +Within a policy, rules are enforced in the order in which they are set. For example, +consider the following: + +- CNP1{tier: Application, priority: 10, ingressRules: [ir1.1, ir1.2], egressRules: [er1.1, er1.2]} +- ANP1{tier: Application, priority: 15, ingressRules: [ir2.1, ir2.2], egressRules: [er2.1, er2.2]} +- CNP3{tier: Emergency, priority: 20, ingressRules: [ir3.1, ir3.2], egressRules: [er3.1, er3.2]} + +This translates to the following order: +- Ingress rules: ir3.1 > ir3.2 > ir1.1 -> ir1.2 -> ir2.1 -> ir2.2 +- Egress rules: er3.1 > er3.2 > er1.1 -> er1.2 -> er2.1 -> er2.2 + +Once a rule is matched, it is executed based on the action set. If none of the +policy rules match, the packet is then enforced for rules created for K8s NP. +Hence, Antrea Policy CRDs take precedence over K8s NP. + ## Notes -- The v1alpha1 CNP CRD supports up to 10000 unique priority at policy level. In - order to reduce the churn in the agent, it is recommended to set the priority - within the range 1.0 to 100.0. +- The v1alpha1 Policy CRDs support up to 10000 unique priority at policy level. + In order to reduce the churn in the agent, it is recommended to set the + priority within the range 1.0 to 100.0.