Skip to content
This repository has been archived by the owner on Apr 16, 2024. It is now read-only.

chore: docker fixes, including clean bootstrap #372

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

edwintorok
Copy link
Contributor

The docker-podman wrapper created volume mounts are owned by the root
user inside the container, and the doom user wouldn't have write access.
Need to specify --user-ns=keep-id flag to map $UID from the host to $UID
from the container without using subuids: that way user inside container
can modify.

SELinux is on by default on Fedora36, thus volume mounts need to specify
the 'Z' flag to relabel the directory being mounted.

I've only tested this on Fedora, applies on top of #371.
Opened as a separate PR since it might need testing on other OSes, to check that the docker command still works there, in particular on non-SELinux systems.

@connorgmeehan
Copy link
Collaborator

Hi @edwintorok, testing this on macos I get the following error. It seems to work if I set --user-ns=auto.

I guess we'll need to detect if the user is using the podman-docker wrapper or just vanilla docker. Do you have any ideas on the best way to do this?

1. Setting up branch
 - Success!  Checked out doom-nvim-contrib branch at:
   /Users/connormeehan/.config/nvim/tools/doom-nvim-contrib

2. Setting up docker environment
 - Docker image does not exist.  Building docker image...
[+] Building 3.5s (18/18) FINISHED
 => [internal] load build definition from Dockerfile                                                                                                                                                                                     0.0s
 => => transferring dockerfile: 37B                                                                                                                                                                                                      0.0s
 => [internal] load .dockerignore                                                                                                                                                                                                        0.0s
 => => transferring context: 2B                                                                                                                                                                                                          0.0s
 => [internal] load metadata for docker.io/library/archlinux:latest                                                                                                                                                                      3.4s
 => [auth] library/archlinux:pull token for registry-1.docker.io                                                                                                                                                                         0.0s
 => [ 1/12] FROM docker.io/library/archlinux:latest@sha256:3a527a8a777472e60c23cf7a610b4d082913a786254c002d1cafdcec7f6129d9                                                                                                              0.0s
 => [internal] load build context                                                                                                                                                                                                        0.0s
 => => transferring context: 38B                                                                                                                                                                                                         0.0s
 => CACHED [ 2/12] RUN PACMAN -Syy                                                                                                                                                                                                       0.0s
 => CACHED [ 3/12] RUN pacman -Sy neovim --noconfirm                                                                                                                                                                                     0.0s
 => CACHED [ 4/12] RUN pacman -Sy ripgrep nodejs-lts-fermium npm git bash gcc jq --noconfirm                                                                                                                                             0.0s
 => CACHED [ 5/12] RUN pacman -Sy wget unzip --noconfirm                                                                                                                                                                                 0.0s
 => CACHED [ 6/12] RUN npm i -g chokidar-cli                                                                                                                                                                                             0.0s
 => CACHED [ 7/12] RUN groupadd doom                                                                                                                                                                                                     0.0s
 => CACHED [ 8/12] RUN useradd -m -g doom doom                                                                                                                                                                                           0.0s
 => CACHED [ 9/12] RUN mkdir /usr/local/lib/node_modules                                                                                                                                                                                 0.0s
 => CACHED [10/12] RUN chown -R doom:doom /usr/local/lib/node_modules/                                                                                                                                                                   0.0s
 => CACHED [11/12] WORKDIR /home/doom                                                                                                                                                                                                    0.0s
 => CACHED [12/12] COPY _docker_entry.sh /usr/local/bin/                                                                                                                                                                                 0.0s
 => exporting to image                                                                                                                                                                                                                   0.0s
 => => exporting layers                                                                                                                                                                                                                  0.0s
 => => writing image sha256:2dd0faf2e2abfbe793e983bb8f7289a7e88378b4b939e2af4c596cf62d3ce967                                                                                                                                             0.0s
 => => naming to docker.io/library/doom-nvim-contrib                                                                                                                                                                                     0.0s

Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them
 - Success! Running docker container doom-nvim-contrib-container...

docker: --userns: invalid USER mode.
See 'docker run --help'.

@edwintorok
Copy link
Contributor Author

Thanks for testing, should be possible to detect podman with $(command -v docker podman | head -n1) and then changing flags accordingly.
I'll do some experiments on Ubuntu (which doesn't have SELinux) and try to update this PR with a script that works on both Ubuntu and Fedora (probably later today/tomorrow).

@edwintorok edwintorok force-pushed the docker-fix-fedora branch 2 times, most recently from 31a410f to 9011cac Compare August 7, 2022 15:42
@edwintorok edwintorok marked this pull request as ready for review August 7, 2022 15:43
@edwintorok
Copy link
Contributor Author

This works on Fedora 36+podman and Ubuntu 22.04+docker now, @connorgmeehan could you test again please and let me know if it works on your system?
I didn't make the :Z conditional for now because Docker understood and ignored that flag.

@edwintorok edwintorok force-pushed the docker-fix-fedora branch 2 times, most recently from 2ad89e4 to e638f46 Compare August 7, 2022 16:07
https://hub.docker.com/_/archlinux/?tab=description says:
"Arch Linux is a rolling release distribution, so a full update is recommended
when installing new packages. In other words, we suggest to either execute RUN
pacman -Syu immediately after your FROM statement or as soon as you docker run
into a container."

Without this there are some errors about LIBC_2.36 when installing new
packages brings in packages depending on it, but the installed libc is
still 2.35.

Signed-off-by: Edwin Török <edwin@etorok.net>
The docker-podman wrapper created volume mounts are owned by the root
user inside the container, and the doom user wouldn't have write access.
Need to specify --user-ns=keep-id flag to map $UID from the host to $UID
from the container without using subuids: that way user inside container
can modify.

SELinux is on by default on Fedora36, thus volume mounts need to specify
the 'Z' flag to relabel the directory being mounted.

podman needs '--userns=keep-id' for permissions of mounted volumes to
work inside the container.
However docker doesn't recognize that flag (and doesn't need it, since
it is running as root).

Detect which of `docker` or `podman` is installed, and if it is podman
add the extra flag. We need to check for podman first, because 'docker'
might just be a wrapper that calls podman.

Signed-off-by: Edwin Török <edwin@etorok.net>
Signed-off-by: Edwin Török <edwin@etorok.net>
Signed-off-by: Edwin Török <edwin@etorok.net>
Signed-off-by: Edwin Török <edwin@etorok.net>
When running doom for the first time (or after ~/.local/share/nvim/site
is cleaned) doom will bootstrap itself by calling packer.sync().
However that happens asynchronously, so do not call DoomStarted in this
case: doom is not ready yet.

This will allow to safely run the following command to synchronize doom
config with packages:
```
nvim --headless --cmd "autocmd User PackerComplete quitall" --cmd "autocmd User DoomStarted PackerSync"
```

Previously such a command wouldn't have been safe on initial run due to
the implicit packer.sync: 2 packer syncs running at the same time would
show a lot of lua module errors about modules not found, it is best
avoided.
On subsequent runs it would've been safe but it is useful to have a
single command to run that is always safe.

This should also make it possible to use a similar command in a CI to
wait for the installation of packages and then test that the doom config
works.

Signed-off-by: Edwin Török <edwin@etorok.net>
Otherwise you get a prompt to pick a registry to pull the image from
(since it failed to build locally).

Signed-off-by: Edwin Török <edwin@etorok.net>
Signed-off-by: Edwin Török <edwin@etorok.net>
@edwintorok edwintorok changed the title chore: make contribute/start_docker work on Fedora 36 chore: docker fixes, including clean bootstrap Dec 31, 2022
@edwintorok
Copy link
Contributor Author

I pushed a few more docker fixes, including a script that can bootstrap nvim from a fresh dir, and another that just deletes the packer_compiled.lua to make testing changes easier.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants