NodePort is always open when using HostEndpoint

[BigKAA]

I'm trying to close the k8s cluster using Calico network policies.
As an example I use https://docs.tigera.io/calico/latest/network-policy/hosts/protect-hosts-tutorial

Expected Behavior

As I understand from the example, NodePort should be unavailable by default. And they should be opened using network policies.

Current Behavior

But, for some reason, all ports reserved for NodePort are open by default. I also cannot close them using policies.

Possible Solution

Steps to Reproduce (for bugs)

1. Add polices

---
apiVersion: crd.projectcalico.org/v1
kind: GlobalNetworkPolicy
metadata:
  name: default-rules
spec:
  selector: node-endpoint == 'home'
  order: 20
  preDNAT: true
  applyOnForward: true
  types:
    - Ingress
  ingress:
    - action: Allow
      source:
        nets:
          - 10.233.0.0/16
#          - 192.168.218.171/32
#          - 192.168.218.174/32
#          - 192.168.218.175/32
#          - 192.168.218.176/32
#          - 192.168.218.177/32
    - action: Allow
      source:
        selector: node-endpoint == 'home'
    - action: Allow
      destination:
        nets:
          - 127.0.0.0/8
    - action: Deny
---
apiVersion: crd.projectcalico.org/v1
kind: GlobalNetworkPolicy
metadata:
  name: allow-outbound-external
spec:
  order: 10
  selector: node-endpoint == 'home'
  types:
    - Egress
  egress:
    - action: Allow
      destination: {}
      source: {}
---
apiVersion: crd.projectcalico.org/v1
kind: GlobalNetworkPolicy
metadata:
  name: allow-rules
spec:
  selector: node-endpoint == 'home'
  order: 10
  preDNAT: true
  applyOnForward: true
  types:
    - Ingress
  ingress:
    - action: Allow
      destination:
        nets:
          - 192.168.218.180/32
        ports:
          - 80
          - 443
      protocol: TCP
    - action: Allow
      destination:
        nets:
          - 192.168.218.189/32
        ports:
          - 7443
      protocol: TCP

2. add Heps

---
apiVersion: crd.projectcalico.org/v1
kind: HostEndpoint
metadata:
  name: control1-ens160
  labels:
    node-endpoint: home
    node-role: control
spec:
  interfaceName: ens160
  node: control1.kryukov.local
  expectedIPs:
    - 192.168.218.171
---
apiVersion: crd.projectcalico.org/v1
kind: HostEndpoint
metadata:
  name: ws1-ens160
  labels:
    node-endpoint: home
    node-role: ws
spec:
  interfaceName: ens160
  node: ws1.kryukov.local
  expectedIPs:
    - 192.168.218.174
---
apiVersion: crd.projectcalico.org/v1
kind: HostEndpoint
metadata:
  name: ws2-ens160
  labels:
    node-endpoint: home
    node-role: ws
spec:
  interfaceName: ens160
  node: ws2.kryukov.local
  expectedIPs:
    - 192.168.218.175
---
apiVersion: crd.projectcalico.org/v1
kind: HostEndpoint
metadata:
  name: ws3-ens160
  labels:
    node-endpoint: home
    node-role: ws
spec:
  interfaceName: ens160
  node: ws3.kryukov.local
  expectedIPs:
    - 192.168.218.176
---
apiVersion: crd.projectcalico.org/v1
kind: HostEndpoint
metadata:
  name: ws4-ens160
  labels:
    node-endpoint: home
    node-role: ws
spec:
  interfaceName: ens160
  node: ws4.kryukov.local
  expectedIPs:
    - 192.168.218.177

3. Some NodePort service

apiVersion: v1
kind: Service
metadata:
  name: app-np
  namespace: app1
spec:
  ports:
    - name: http
      protocol: TCP
      port: 80
      targetPort: http
      nodePort: 31110
  selector:
    app.kubernetes.io/instance: app1
    app.kubernetes.io/name: uniproxy
  type: NodePort

4. On none k8s cluster host

curl http://192.168.218.171:31110

Context

I tried to explicitly describe the protected ports in the FailsafeInboundHostPorts parameter in FelixConfiguration. Not including ports reserved for NodePort. Did not help.

Your Environment

• Calico version 3.26.3
• Kubernetes: v1.26.9 v1.28.2
  • IPVS
• Operating System and version: AlmaLinux 8.8 (ID_LIKE="rhel centos fedora")
• Link to your project (optional):
  • https://github.com/BigKAA/youtube/blob/hep/net/05-NetworkPolicy-calico/hep/cluster-heps.yaml
  • https://github.com/BigKAA/youtube/blob/hep/net/05-NetworkPolicy-calico/np/np-10.yaml
  • https://github.com/BigKAA/youtube/blob/hep/net/05-NetworkPolicy-calico/images/cluster.jpg

[caseydavenport]

apiVersion: crd.projectcalico.org/v1

I'm obliged to link this issue explaining Calico API versions: #6412

On none k8s cluster host

Sorry - it's not obvious to me from the issue, but where are you running this curl command from - is it from one of your Kubernetes nodes or somewhere else?

[BigKAA]

Sorry - it's not obvious to me from the issue, but where are you running this curl command from - is it from one of your Kubernetes nodes or somewhere else?

"somewhere else". On any hosts outside of k8s cluster

[BigKAA]

I changed the API in the manifests at projectcalico.org/v3. Did not help.

https://github.com/BigKAA/youtube/blob/hep/net/05-NetworkPolicy-calico/np/np-10.yaml

Access to the NodePort service remains open.

[BigKAA]

Very strange. When requesting a control node of the cluster, the node port is not closed. And on the worker node of the cluster it is closed.

[mazdakn]

@BigKAA have you checked the failsafe ports?

[BigKAA]

@BigKAA have you checked the failsafe ports?

@mazdakn Yes, I checked. Separately, I made my own list of ports to check.

apiVersion: projectcalico.org/v3
kind: FelixConfiguration
metadata:
  name: default
spec:
  bpfLogLevel: ''
  floatingIPs: Disabled
  healthPort: 9099
  logSeverityScreen: Info
  reportingInterval: 0s
  failsafeInboundHostPorts:
    - "tcp:22"
    - "udp:68"
    - "tcp:179"
    - "tcp:2379"
    - "tcp:2380"
    - "tcp:6443"
    - "tcp:6666"
    - "tcp:6667"

In this video I show the problem. Sorry, the video is in Russian. https://youtu.be/5g2cnLvgjq8?si=bZvpnPW8qkG47cmg&t=2283

[tomastigera]

@BigKAA have you made any progress? Did you check how the policies are rendered in iptables/ipvs rules? Did you try to apply policies step by step? Have you tried to deny everything from outside with a simpler policy? In what CIDR is your client?

[BigKAA]

@tomastigera I stopped using Calico policies to restrict external connections. Switched to classic iptables