Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable sudo nerdctl run to expose ports to localhost #242

Merged
merged 6 commits into from
Sep 17, 2021

Conversation

mattfarina
Copy link
Contributor

A couple changes were made to make this possible including:

  • Adding iptables package to read the ports exposed by portmap
    CNI plugin. This was done so that 1 call to iptables is all
    that is needed on each run.
  • Move the lima-guestagent to run as root. This is needed for
    permission to use iptables.

Note, this implementation only includes ipv4 support. Does not work for ipv6.

Closes #140

Note, this is my first pass. Happy to make changes as needed.

@jandubois jandubois added enhancement New feature or request bug Something isn't working and removed enhancement New feature or request labels Sep 14, 2021
@AkihiroSuda AkihiroSuda added this to the v0.7.0 milestone Sep 14, 2021
// ipv4 IP address. We need to detect this IP.
// --dport is the destination port. We need to detect this port
// -j DNAT this tells us it's the line doing the port forwarding.
var findPortRegex = regexp.MustCompile(`-A\s+CNI-DN-\w*\s+(?:-d ((?:\b25[0-5]|\b2[0-4][0-9]|\b[01]?[0-9][0-9]?)(?:\.(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)){3}))?(?:/32\s+)?-p .*--dport (\d+) -j DNAT`)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should limit the regex to CNI rules.
We should cover non-CNI rules such as Docker libnetwork rules with userland-proxy: false.

We can just overestimate the set of the listening ports, and probably try connect()-ing to verify whether the port is actually open (for TCP ports).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested lima with Docker and userland-proxy: false. The port forwarding with the latest release of lima and this patch works without any changes. The iptables rule for forwarding is listed and reads...

-A DOCKER -p tcp -m tcp --dport 8081 -j DNAT --to-destination 172.17.0.2:80

When the userland proxy is in place the iptables rule is different...

-A DOCKER -d 127.0.0.1/32 ! -i docker0 -p tcp -m tcp --dport 8081 -j DNAT --to-destination 172.17.0.2:80

Docker (I think moby) is doing something to make the ports work. For example, running netstat -plunt will show the forwarded ports for docker but not containerd (running as root).

Do we need to support Docker if it already works?

Note, I tested docker running rootless and as root.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I updated the iptables check to verify TCP ports are open before returning them.

@@ -17,11 +17,6 @@ install -m 755 "${LIMA_CIDATA_MNT}"/lima-guestagent /usr/local/bin/lima-guestage

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably need to uninstall the previous version of the guest agent

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably need to uninstall the previous version of the guest agent

Just out of curiosity: why do we need to remove it? install should be able to overwrite the file. What am I missing?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not overwritten because we switched away from non-root to root

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I meant we have to remove non-root systemd unit

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I meant we have to remove non-root systemd unit

Ok, thanks, because that is not what mattfarina@3aba8f3 is doing...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes sense. Just pushed a change to remove the legacy systemd configuration.

@@ -40,16 +28,13 @@ var daemonCommand = &cli.Command{
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably this ticking is too heavy for watching iptables. Instead we should wait for NETLINK_AUDIT events or some eBPF stuff, but I can work on that in a separate PR after merging this.

Copy link
Member

@AkihiroSuda AkihiroSuda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, LGTM but let me release v0.6.4 before merging this

A couple changes were made to make this possible including:
- Adding iptables package to read the ports exposed by portmap
  CNI plugin. This was done so that 1 call to iptables is all
  that is needed on each run.
- Move the lima-guestagent to run as root. This is needed for
  permission to use iptables.

Note, this implementation only includes ipv4 support. Does not
work for ipv6.

Signed-off-by: Matt Farina <matt.farina@suse.com>
Signed-off-by: Matt Farina <matt.farina@suse.com>
Signed-off-by: Matt Farina <matt.farina@suse.com>
The -f flag is used on rm to ensure no returned error if the file
is not found. This would happen on the first run.

Signed-off-by: Matt Farina <matt.farina@suse.com>
Signed-off-by: Matt Farina <matt.farina@suse.com>
Signed-off-by: Matt Farina <matt.farina@suse.com>
@mattfarina
Copy link
Contributor Author

The latest push is a rebase on master after the cobra changes landed.

Copy link
Member

@AkihiroSuda AkihiroSuda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working impact/changelog
Projects
None yet
Development

Successfully merging this pull request may close these issues.

port forwarding does not work for sudo nerdctl (i.e. rootful CNI portmap)
3 participants