fatal: unable to exec run: Permission denied

[mazzystr]

Host OS:

# cat /etc/redhat-release
CentOS Linux release 7.6.1810 (Core)

Container info:

# podman image ls nzbget
REPOSITORY                                   TAG      IMAGE ID       CREATED       SIZE
registry.hub.docker.com/linuxserver/nzbget   latest   309400c08290   10 days ago   102 MB

Cmd:

# /usr/bin/podman run -it \
  --name nzbget \
  --hostname nzbget \
  --net=host \
  -p 9081:9081 \ 
  -e PUID=666 -e PGID=666 \
  -v /mnt/daemons/nzbget/config:/config \
  -v /mnt/daemons/nzbget/downloads:/downloads \
  linuxserver/nzbget

Log:

-------------------------------------
GID/UID
-------------------------------------

User uid:    666
User gid:    666
-------------------------------------

[cont-init.d] 10-adduser: exited 0.
[cont-init.d] 30-config: executing...
[cont-init.d] 30-config: exited 0.
[cont-init.d] done.
[services.d] starting services
s6-supervise (child): fatal: unable to exec run: Permission denied
s6-supervise nzbget: warning: unable to spawn ./run - waiting 10 seconds
[services.d] done.
s6-supervise (child): fatal: unable to exec run: Permission denied
s6-supervise nzbget: warning: unable to spawn ./run - waiting 10 seconds
s6-supervise (child): fatal: unable to exec run: Permission denied
s6-supervise nzbget: warning: unable to spawn ./run - waiting 10 seconds
s6-supervise (child): fatal: unable to exec run: Permission denied
s6-supervise nzbget: warning: unable to spawn ./run - waiting 10 seconds

[aptalca]

podman? is that some custom docker service build?

[mazzystr]

oh my dood ... welcome to a world without docker process, binary or company! Allow me to introduce to you my friend podman.

It's looking like my old conf file has a bunch of directories in values that aren't compatible with this container. I'm still investigating.

[aptalca]

start with an empty config folder to test

[mazzystr]

I got it working with minimal hacking

The container needs this softlink...
# ln -s /downloads /app/nzbget/downloads

If you do that I'll make a deal with you and adjust my conf to...

SecureCert=/config/local.crt
SecureKey=/config/local.key

We should really figure out how to make this container a seamless drop in replacement for those coming in from the host/process world.

[aptalca]

Why?

Just set the download location to /downloads in the app settings?

If you really really really need to use that location, map your downloads folder on the host to /app/nzbget/downloads inside the container

[thelamer]

@mazzystr glad you got it figured out , from a support standpoint these are pushed from DockerHub to be used with Docker. Anything outside of this , we do not have the bandwidth to test for. We do manage build logic across 100+ repos and are not the largest team.

[mazzystr]

Guys, closing out an issue before a user is satisfied uncool... Don't be that community.

Podman is being pushed by the Open Container Initiative to bypass Docker company shenanigans. Ya'll best be keeping current on the topic.

I like the -v /mnt/daemons/nzbget/downloads:/app/nzbget/downloads. Good catch on that. This should be noted as default in the documentation. Again people coming in from the host/process world are going to expect a seamless move into this container.

We're not done yet tho. This log...

s6-supervise (child): fatal: unable to exec run: Permission denied
s6-supervise nzbget: warning: unable to spawn ./run - waiting 10 seconds

is still causing initialization problems. The daemon is not running at this time. I'm forced to interact with the container and execute exec s6-setuidgid abc /app/nzbget/nzbget -s -c /config/nzbget.conf -o OutputMode=log. I'm still investigating and I'll happily submit a PR when I find the culprit.

[aptalca]

Well, you did say I got it working with minimal hacking

[thelamer]

@mazzystr yeah but we are not going to accept that PR. Until we have some kind of initiative to support alternative container platforms any modfications to the init logic or build process would need to be completely unobtrusive to a standard Docker User and would likely be seen as excess unneeded logic.

When you say people migrating from host level processes, even that is not officially supported. If you want to mount up your configs from another instance of nzbget that we did not compile (literally) and bypass our init process to put a clean tested known working configuration that is on your plate.

We as an organization for the time being have an extreme Bias towards our users using Docker, as we have become heavily entrenched in their ecosystem and associated sdk tools. This is not just an afternoon project to get a single container working you are discussing, for us it is an overhaul to tens of thousands of lines of code spanning many repos and local build servers.

I hope you understand why :

1. This is not the place to discuss these kinds of changes please hop on our forums or discord if this is not a direct bug report or using our container in the manner we recommend.
   https://discourse.linuxserver.io/
   https://discord.gg/YWrKVTn
2. That supporting alternative container platforms is not out of our mind, but we have more pressing tasks with the bandwidth that we currently have at our disposal.

[mazzystr]

You got me with the literalness :D

It looks like there is an upstream issue with the Alpine container. We're trying to run this process as a non-uid 0 uid. The process needs to put data into /run but /run has the following permissions...

$ ls -ld /run
drwxr-xr-x 5 root root 120 Feb 22 20:47 /run

thus the permission denied log. Lots of other Fedora/CentOS users are having this issue using docker, podman and CRI-O runtimes.

This workaround is acceptable...

mkdir /run/nzbget
chown ${uid}:${uid} /run/nzbget
/usr/bin/podman run -d --name nzbget --hostname nzbget -p ${SecurePort}:${SecurePort} -e PUID=${uid} -e PGID=${uid} -v ${AppDir}:/config -v ${AppDir}/downloads:/app/nzbget/downloads -v /run/nzbget:/run linuxserver/nzbget

[mazzystr]

App owners resisted virtualization at one time too. It's in the very nature of dev and ops teams to resist change. It's why issues are triggered. Us users just try to get on.

I'm now good with closing out this issue. Have a good day gents.

[liamdawson]

For future readers (as this was high up in the search results): this is a conflict between s6-overlay and podman, triggered by the name of the entrypoint (init).

Set --systemd=false to prevent the mounting behaviour that causes this conflict. (Source)

[ubellavance]

@thelamer I think it may be a good idea to add this to the documentation... I just ran successfully your nice unifi-controller container using podman and all I needed is --systemd=false

[aptalca]

@ubellavance that's really a podman specific setting (kind of silly for them to assume every container uses regular init, or maybe I'm not aware of a better reasoning they may have). Anyway, we don't officially support podman or any other docker fork currently. We only officially support docker installed from the official docker repo.

I believe podman should be promoting that instead as their system doesn't work with any image that uses s6 (or perhaps other hypervisors as well) out of the box.

[ubellavance]

@aptalca thanks for your reply. AFAIK podman people (mostly Red Hat, BTW) simply have goals that are different from docker, and they therefore have different design and restrictions. And it's not a fork, it's a complete rewrite and one of the positive parts of it is that can run without a daemon.

I can totally understand that you don't want to support podman, I only asked to add a note somewhere in the documentation. It could be an opportunity for linuxserver.io to state more clearly that only upstream docker is officially supported but say that you have reports that your containers can run with podman with --systemd=false. I also think that you should have a look at podman/buildah/skopeo, but I understand your time is limited.

[thelamer]

It is not a time thing, it is a round peg and a square hole. We as an organization make Docker containers, that is what we do.
We recommend you use Docker , you want to use something else go ahead , but you get zero and I mean zero support.
If you would like to fork our images or join an organization surrounding podman go ahead, the source is there and is not restrictively licensed.
We will not be adding notes for non Docker platforms , well at least I would vote against it if brought up.

[CHBMB]

FWIW I agree with @thelamer we just can't be all things to all men (& women), we don't have the resources in terms of either manpower or infra to support a completely different container runtime.

[aleks-mariusz]

Apologies upfront for beating a dead horse here, but I wanted to chime in with another perspective, due to the needless anti-podman sentiment here. It might have been missed but one of the things that makes podman wonderful is that it does pretty much everything docker does, but more securely. No daemon needed, no root access required, It's like docker but re-written w/ lessons learned since docker came into creation, designed to be a drop-in replacement for docker and so all docker related commands work just about identically.

Docker (as a technology) really only uses two main kernel features, cgroups and namespaces.. While they were the pioneers of using these features to bring containerization to the masses, these days, utilities like podman, and it's bretheren skopeo/buildah do the exact same thing, without requiring an insecure bloated daemon to do what you can do directly w/ those kernel features. AND podman can do the same magic without requiring root (while docker is still fairly new in trying to be 'rootless'). Perhaps some-more-research would be helpful, as it's the general direction the container ecosystem is likely heading...

Why? well, Docker (the company) has been acquired late last year, so it's likely docker's days are numbered (even if they still amount to months if not years). It doesn't seem sustainable for docker to provide much more value than it already has, sure it's heavily entrenched, but its technological value kind of peaked around 2017/2018 and now it's no longer really needed. While I agree it's unlikely dockerhub is going anywhere, flat out refusing to even add a note on your README and mention podman, just seems unhelpful. Good thing google helped me find the answer at least despite that stubbornness.

[CHBMB]

@aleks-mariusz There is a question of (voluntary) man power here. We just can't support everything. Now if some people wanted to volunteer their free time and become the podman lead for the organisation, including fielding all the support and doing all the testing, then that could change?

[aleks-mariusz]

Hmm.. well if you're offering me to volunteer for this role/asking me for help with that, then I'm intrigued.. BUt i'm afraid I don't know a whole lot about LinuxServer.io other than the awesome containers you guys put out. If you have a link to the bits i'm unclear on (what the process is to get onboarded as a podman lead, what you mean by the testing, beyond just making sure containers start with podman, in general to have an idea of what kind of time commitment this will require) then i can consider it.

If you were just generally speaking about needing more manpower to support podman, fair enough, I can file a PR with the README change but i've been given the impression that it'd just be shot down?

[CHBMB]

No, what I'm saying is, if we do it for one container, we'd need to do it for all. Including changing all the CI, to confirm it works as expected with podman.

Then, as none of the current members use podman, someone would need to step up to support any podman queries, across our forum, Discord and Github going forward.

It's a huge undertaking and most of our members are already spending every bit of free time with the docker stuff.

The reason we're resistant to even adding a note to the readme, is that it implies some level of support. "Your readme has a podman command"

We already tell people we don't support Mac or Windows as hosts.

I think the problem is, one guy/girl has an issue with podman, and we currently have 0 interest in supporting it, or even implying that we support it. If, and that's a big if, we ever decided to change that, it's a massive amount of work to do so.

This is far far bigger than a PR to a README

[aptalca]

Also, I think you're mistaking our indifference for needless anti-podman sentiment. We are a volunteer FOSS group with a very specific mission and a set of goals. Those goals at the moment do not include podman, buildah, or whatever other docker alternative there is.