Commands executing in Scuba can't write to /dev/stdout

[xanarin]

I was writing a bash script that will run inside of a docker container launched with scuba. In that script, I use a heredoc in order to write my help message to STDOUT. The function looks like this:

function show_help() {
    cat > /dev/stdout << END
USAGE: ${0} [-f] [-c] [-h]

OPTIONAL ARGS:
    -f   Run tests in "failfast" mode, where testing stops on first failed or errored test
    -c   Measure coverage during testing and output coverage results
    -v   Verbose output
    -h   Display this help and exit
END
}

If I run this script on my host, it works correctly. However, if I run it under Scuba, I get the following error:

/bin/bash: /dev/stdout: Permission denied

I did some research, and I think the issue is the same as the one addressed in this SO post: bash:/dev/stderr: Permission denied. Additionally, I can see that the permissions of /dev/stdout (once all symlinks are followed) are 0620 and the file is owned by root:tty:

$ scuba /bin/bash -c "ls -lH /dev/stdout"
crw--w---- 1 root tty 136, 0 Mar 12 19:27 /dev/stdout

I am able to minimally reproduce this issue by running the following command:

$ scuba /bin/bash -c "echo 'foobar' > /dev/stdout"

[JonathonReinhart]

Manually walking the symlinks, we find that /dev/pts/0 (the pseudoterminal slave created by Docker) has the problematic permissions. But with Docker, this isn't a problem, even if we specify -u to run as a separate user:

$ scuba /bin/sh -c "ls -l /dev/pts/0; id > /dev/stdout"
crw--w---- 1 root tty 136, 0 Mar 12 20:00 /dev/pts/0
/bin/sh: 1: cannot create /dev/stdout: Permission denied

$ docker run --rm -it debian:8 /bin/sh -c "ls -l /dev/pts/0; id > /dev/stdout"
crw--w---- 1 root tty 136, 0 Mar 12 20:02 /dev/pts/0
uid=0(root) gid=0(root) groups=0(root)

$ docker run --rm -it -u 1000 debian:8 /bin/sh -c "ls -l /dev/pts/0; id > /dev/stdout"
crw--w---- 1 1000 tty 136, 0 Mar 12 20:02 /dev/pts/0
uid=1000 gid=0(root) groups=0(root)

It looks like the problem is this:

• Docker creates the PTS as the same user that the container runs as
• But scuba runs the container as root, and then scubauser drops to the desired user id

Ugh.

At first glance, it seems like scubainit needs to change permissions on (the ultimate symlink target of) /dev/stdout (and probably /dev/stdin, /dev/stderr) before exec'ing the child process.

[xanarin]

The following short-term workaround seems to work for me and I'm planning on using it until this issue is resolved:

hooks:
  root: chown scubauser:tty /dev/pts/0

(This solution was suggested to me by @JonathonReinhart)

[JonathonReinhart]

On another machine running docker-1.13.1-65.git1185cfd.fc28.x86_64, I see different symlinks:

$ docker run --rm -it debian:8 ls -l /dev/stdout
lrwxrwxrwx. 1 root root 15 Mar 13 13:33 /dev/stdout -> /proc/self/fd/1

$ docker run --rm -it debian:8 ls -l /proc/self/fd
total 0
lrwx------. 1 root root 64 Mar 13 13:33 0 -> /18
lrwx------. 1 root root 64 Mar 13 13:33 1 -> /18
lrwx------. 1 root root 64 Mar 13 13:33 2 -> /18
lr-x------. 1 root root 64 Mar 13 13:33 3 -> /proc/1/fd

Related issues:

• /dev/stdout is inaccessible by an unprivileged user in a container started as root moby/moby#31243
• unable to modify permissions of /dev/stdout without tty moby/moby#31106 (comment)
• /proc/self/fd/{0,1,2} are not valid (with --tty) moby/moby#11462
• Permission denied on /dev/stderr moby/moby#6880

[JonathonReinhart]

@xanarin An even easier workaround is to just remove the redundant > /dev/stdout redirection:

function show_help() {
    cat << END
USAGE: ${0} [-f] [-c] [-h]

OPTIONAL ARGS:
    -f   Run tests in "failfast" mode, where testing stops on first failed or errored test
    -c   Measure coverage during testing and output coverage results
    -v   Verbose output
    -h   Display this help and exit
END
}

[JonathonReinhart]

@xanarin Here's a WIP branch you can play with and/or review: #131

[xanarin]

In an offline discussion with @JonathonReinhart we wondered if there was a way to reproduce this behavior outside of docker. It turns out there is. Here are 3 PoC snippets that show similar behavior on a normal Linux system with no containers involved. In these snippets, dummy is a user with very low privileges, and the bash session is running as user smartguy:

$ sudo -u dummy /bin/bash -c 'cat > /dev/stdout << END
FOOBAR
END'
/bin/bash: /dev/stdout: Permission denied

$ echo "foo" > bar
$ sudo -u dummy /bin/bash -c 'cat bar > /dev/stdout'
/bin/bash: /dev/stdout: Permission denied

$ sudo -u dummy /bin/bash -c 'echo foo | tee /dev/stdout'
tee: /dev/stdout: Permission denied
foo

I believe that these permissions issues occur when a running process launches a child process that changes its UID/EUID, then tries to explicitly open /dev/stdout or /dev/stderr:

$ id
uid=1000(smartguy) gid=1000(smartguy) groups=1000(smartguy),10(wheel),91(video)

$ sudo -u dummy ls -lH /dev/stdout
crw--w---- 1 smartguy tty 136, 6 Mar 25 16:59 /dev/stdout