Rootless buildah + slirp4netns = no network

[matpen]

Description

I am trying to get buildah and podman to work in rootless mode. With the default install (Ubuntu 16.04 via PPA) podman complains about missing slirp4netns. However, after installing the library buildah run has no network access.

I found a workaround by adding --network=host to the buildah run command. However, I still feel that this issue should be brought to the attention of the devs.

Steps to reproduce the issue:

I am performing the following minimal example to reproduce the behavior. Note that the steps are performed without slirp4netns installed and later with it. Removing the library restores consistently the initial behavior.

1. Make a new working container

container="$(buildah from docker://docker.io/library/ubuntu:16.04)"

2. Run a command which requires network access, for example apt-get. Note that, since I am running buildah 1.8-dev, I am affected by this bug and therefore applying this workaround

BUILDAH_ISOLATION=rootless buildah run --isolation=  \
    "$container" -- \
    bash -c 'apt-get -yq update'

3. Commit the container to an image

buildah commit --rm "$container" "test-image"

4. Run a podman container based on the image. If slirp4netns is not installed, podman gives error (see comments below and here) so I installed it with apt-get install slirp4netns (which leads to the problem described in this issue).

# Without slirp4netns: time="..." level=error msg="could not find slirp4netns, the network namespace won't be configured: exec: "slirp4netns": executable file not found in $PATH"
# With slirp4netns: all good
podman run --rm="true" --name="meta.build" "test-image" ls /

Describe the results you received:

Step 2 above:

• works normally without slirp4netns
• does not work with slirp4netns (container has no network access)
• works even with slirp4netns if run as root (replace BUILDAH_ISOLATION=rootless buildah run --isolation= with sudo buildah run
• works even with slirp4netns if run as normal user and adding the option --network=host to the buildah run command

Describe the results you expected:

Step 2 should probably "just work". Buildah should detect the rootless environment and adjust as necessary.

Also, if podman requires slirp4netns it should be probably added as a dependency of the PPA package.

Output of rpm -q buildah or apt list buildah:

Listing... Done
buildah/xenial,now 1.8-1~dev~ubuntu16.04.2~ppa5 amd64 [installed]

Output of buildah version:

Version:         1.8-dev
Go Version:      go1.10.4
Image Spec:      1.0.0
Runtime Spec:    1.0.0
CNI Spec:        0.4.0
libcni Version:  
Git Commit:      
Built:           Sat Apr 13 18:42:38 2019
OS/Arch:         linux/amd64

Output of podman version if reporting a podman build issue:

Version:            1.3.2-dev
RemoteAPI Version:  1
Go Version:         go1.10.4
OS/Arch:            linux/amd64

Output of cat /etc/*release:

DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04.6 LTS"
NAME="Ubuntu"
VERSION="16.04.6 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.6 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial

Output of uname -a:

Linux laptop 4.4.0-150-generic #176-Ubuntu SMP Wed May 29 18:56:26 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Output of cat /etc/containers/storage.conf:

# storage.conf is the configuration file for all tools
# that share the containers/storage libraries
# See man 5 containers-storage.conf for more information

# The "container storage" table contains all of the server options.
[storage]

# Default Storage Driver
driver = "overlay"

# Temporary storage location
runroot = "/var/run/containers/storage"

# Primary read-write location of container storage
graphroot = "/var/lib/containers/storage"

[storage.options]
# AdditionalImageStores is used to pass paths to additional read-only image stores
# Must be comma separated list.
additionalimagestores = [
]

# Size is used to set a maximum size of the container image.  Only supported by
# certain container storage drivers (currently overlay, zfs, vfs, btrfs)
size = ""

# OverrideKernelCheck tells the driver to ignore kernel checks based on kernel version
override_kernel_check = "true"

[matpen]

Might containers/podman#3277 be related?

[rhatdan]

Seems reasonable.

@giuseppe PTAL

[giuseppe]

we should be using --network=host by default now, also when running as rootless.

How does your /etc/resolv.conf look like?

[matpen]

we should be using --network=host by default now, also when running as rootless.

Is this on 1.8.1-dev too, or more recent release? As soon as I can get a chance, I want to try out the latest 1.9.0

How does your /etc/resolv.conf look like?

Just the default one on my workstation: classic DHCP home router

# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 127.0.1.1
search lan

[rhatdan]

That would be the issue. Since the no process is listening at 127.0.1.1

[matpen]

That would be the issue. Since the no process is listening at 127.0.1.1

Sorry, I lost you there. Are you saying that the conflict is caused by the DHCP daemon? As far as I know, there is nothing special about it on the system I am testing on.

Could you suggest a possible workaround?

[giuseppe]

127.0.1.1 inside the container won't access the process that is probably running on your host. We would need to generate the /etc/resolv.conf file in the same way as we do with Podman. I'd suggest you to either use --net host or bind mount a /etc/resolv.conf containing DNS servers accessible from the container.

[rhatdan]

@giuseppe Can you port over the way we handle this in Podman to Buildah?

[giuseppe]

what would be the best way to share code? Should we move the generation of /etc/resolv.conf to buildah and use it from libpod?

@mheon

[matpen]

@giuseppe Thank you for the explanation, it is clear now. Using --net host indeed works for me.

[rhatdan]

@giuseppe Yes move it to a pkg under buildah.