Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BPF] BPFExcludeCIDRsFromNAT allows excluding service IPs from being NATed #8338

Merged

Conversation

tomastigera
Copy link
Contributor

@tomastigera tomastigera commented Dec 14, 2023

The main use case is node-local dns cache. In iptables mode with k8s provided kube-proxy, the cache installs such iptables rules that snatch the traffic before the kube-proxy rules, which would do the NAT.

However, in ebpf NAT is done way before packets may hit iptables. Often they do not hit iptables at all. Therefore, we need to tell our kube-proxy, not to translate a service IP that we want to let dns cache to process.

Unfortunatelly, we cannot annotate the service as the annotations are not propagate to the kube-proxy code by the k8s front-end.

To setup a node-local dns cache, set BPFExcludeCIDRsFromNAT as follows:

kubedns=kubectl get svc kube-dns -n kube-system -o jsonpath={.spec.clusterIP}

BPFExcludeCIDRsFromNAT=$kubedns

fixes #4689

Description

Related issues/PRs

Todos

  • Tests
  • Documentation
  • Release note

Release Note

ebpf: setting BPFExcludeIPsFromNAT allows node-local dns cache to work

Reminder for the reviewer

Make sure that this PR has the correct labels and milestone set.

Every PR needs one docs-* label.

  • docs-pr-required: This change requires a change to the documentation that has not been completed yet.
  • docs-completed: This change has all necessary documentation completed.
  • docs-not-required: This change has no user-facing impact and requires no docs.

Every PR needs one release-note-* label.

  • release-note-required: This PR has user-facing changes. Most PRs should have this label.
  • release-note-not-required: This PR has no user-facing changes.

Other optional labels:

  • cherry-pick-candidate: This PR should be cherry-picked to an earlier release. For bug fixes only.
  • needs-operator-pr: This PR is related to install and requires a corresponding change to the operator.

@marvin-tigera marvin-tigera added this to the Calico v3.28.0 milestone Dec 14, 2023
@marvin-tigera marvin-tigera added release-note-required Change has user-facing impact (no matter how small) docs-pr-required Change is not yet documented labels Dec 14, 2023
@tomastigera tomastigera force-pushed the tomas-bpf-node-local-dns branch 4 times, most recently from b5c7cb6 to 3e0c863 Compare December 15, 2023 21:19
@tomastigera tomastigera marked this pull request as ready for review December 16, 2023 04:03
@tomastigera tomastigera requested a review from a team as a code owner December 16, 2023 04:03
api/pkg/apis/projectcalico/v3/felixconfig.go Outdated Show resolved Hide resolved
api/pkg/apis/projectcalico/v3/felixconfig.go Outdated Show resolved Hide resolved
felix/dataplane/linux/bpf_ep_mgr.go Outdated Show resolved Hide resolved
felix/bpf-gpl/tc.c Show resolved Hide resolved
…g NATed

The main use case is node-local dns cache. In iptables mode with k8s
provided kube-proxy, the cache installs such iptables rules that snatch
the traffic before the kube-proxy rules, which would do the NAT.

However, in ebpf NAT is done way before packets may hit iptables. Often
they do not hit iptables at all. Therefore, we need to tell our
kube-proxy, not to translate a service IP that we want to let dns cache
to process.

Unfortunatelly, we cannot annotate the service as the annotations are
not propagate to the kube-proxy code by the k8s front-end.

To setup a node-local dns cache, set BPFExcludeCIDRsFromNAT as follows:

kubedns=`kubectl get svc kube-dns -n kube-system -o jsonpath={.spec.clusterIP}`

BPFExcludeCIDRsFromNAT=$kubedns
@tomastigera tomastigera force-pushed the tomas-bpf-node-local-dns branch from 3e0c863 to 5b3058a Compare December 20, 2023 19:19
@tomastigera tomastigera changed the title [BPF] BPFExcludeIPsFromNAT allows excluding service IPs from being NATed [BPF] BPFExcludeCIDRsFromNAT allows excluding service IPs from being NATed Dec 20, 2023
@tomastigera
Copy link
Contributor Author

/merge-when-ready

@marvin-tigera
Copy link
Contributor

OK, I will merge the pull request when it's ready, leave the commits as is when I merge it, and leave the branch after I've merged it.

@marvin-tigera marvin-tigera merged commit 6691a82 into projectcalico:master Dec 21, 2023
2 checks passed
tomastigera added a commit that referenced this pull request Feb 2, 2024
…v3.27

[release-v3.27] Auto pick #8338: BPFExcludeCIDRsFromNAT allows excluding service CIDRs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs-pr-required Change is not yet documented merge-when-ready release-note-required Change has user-facing impact (no matter how small)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

NodeLocal DNS with Calico/BPF
3 participants