Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sch_netem kernel module missing for network emulation (tc netem) #6033

Closed
LaumiH opened this issue Dec 9, 2019 · 3 comments · Fixed by #6038
Closed

sch_netem kernel module missing for network emulation (tc netem) #6033

LaumiH opened this issue Dec 9, 2019 · 3 comments · Fixed by #6038
Assignees
Labels
good first issue Denotes an issue ready for a new contributor, according to the "help wanted" guidelines. help wanted Denotes an issue that needs help from a contributor. Must meet "help wanted" guidelines. kind/feature Categorizes issue or PR as related to a new feature. priority/backlog Higher priority than priority/awaiting-more-evidence.

Comments

@LaumiH
Copy link
Contributor

LaumiH commented Dec 9, 2019

The exact command to reproduce the issue:
Application based: Run a container like debian in priviledged mode in a pod and one pod that uses the netem network emulation linux tool, in my case pumba. Pumba has no effect trying to mess with the network of the debian container. Pumba uses netem.

modprobe: Try to run minikube ssh and then modprobe -n sch_netem, which says netem module is missing.

With netem on a priviledged debian container in minikube: Get a bash in the debian container and try to modify the eth0 virtual interface by changing qdisc to netem loss (which is essentially what pumba does). The command is tc qdisc add dev eth0 root netem loss random 100%.

A related issue, and the environment in which I found out this is not working, is described here. This is a minikube specific issue, as it works with other kubeadm based clusters and GKE and others. It also works locally on a laptop just with docker images.

The full output of the command that failed:

  • modprobe -n sch_netem -> modprobe: FATAL: Module sch_netem not found in directory /lib/modules/4.19.76
  • tc qdisc add dev eth0 root netem loss random 100 -> invalid qdisc name

The operating system version:

  • 5.4.1-2-MANJARO
  • minikube version: v1.5.2
    commit: 792dbf9

Additional info
I just wanted to let you know this issue exists. Maybe we can do something about it, as minikube is a great environment for starting with chaos engineering, and not being able to mess with the network (in this way) is a big drawback. Also it took me quite some time to figure this out, as I had no access to other clusters :D.

@tstromberg
Copy link
Contributor

Sounds reasonable. Here are the docs for adding a new kernel module to the ISO if anyone is interested in seeing this through:

https://minikube.sigs.k8s.io/docs/contributing/iso/#adding-kernel-modules

@tstromberg tstromberg added good first issue Denotes an issue ready for a new contributor, according to the "help wanted" guidelines. help wanted Denotes an issue that needs help from a contributor. Must meet "help wanted" guidelines. kind/feature Categorizes issue or PR as related to a new feature. priority/backlog Higher priority than priority/awaiting-more-evidence. labels Dec 9, 2019
@LaumiH
Copy link
Contributor Author

LaumiH commented Dec 9, 2019

Going to PR the change as soon as my slow network is willing to clone the repo ...
Apply minikube ssh and tc qdisc add dev eth0 root netem loss random 100% to see how magnificently it works. But don't cry when the ssh connection fails because of 100% network loss ^^.

@LaumiH
Copy link
Contributor Author

LaumiH commented Dec 10, 2019

/assign

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Denotes an issue ready for a new contributor, according to the "help wanted" guidelines. help wanted Denotes an issue that needs help from a contributor. Must meet "help wanted" guidelines. kind/feature Categorizes issue or PR as related to a new feature. priority/backlog Higher priority than priority/awaiting-more-evidence.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants