Docker image improvements #8
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Clean up the files at the end to shorten build time as package sources don't have to be repopulated, and the final image is truly cleaned. Also clean some more based on some references. It's less than 50MB, which is nothing compared to a 1.27GB image, but it's a good practice.
RetroPie-Setup take 1-2 hours. Minimize what is done before it by moving every non-essential package install to after.
Pre-installing packages installed by retro-cloud might speed up the testing, but makes it less reliable.
They're only there for RetroPie-Setup.
Mimicks the "pi" user on a RPi.
`useradd` is generally recommended for scripted used creation, but despite the effort of multiple calls to get the correct access rights etc, the user still didn't belong to the right group id etc. Using `adduser` as recommended by Raspberry Pi docs and RetroPie-Setup author worked on the first attempt.
Basic bash configs taken from a RetroPie 4.5.1 on a RaspberryPi 3. They are copied over when creating a new user. See: https://www.raspberrypi.org/documentation/linux/usage/users.md
Was installing the Raspbian packages according to: https://retropie.org.uk/docs/Manual-Installation/ But this image is actually Debian, built on Windows/Linux.
It's called in the beginning when pulling in the base image. It only saves a couple of seconds but should keep cache layers before RetroPie-Setup minimal.
This wasn't noticed before because the Docker image had it pre-installed to speed up the testing.
seriema
commented
Mar 19, 2020
| # These files were copied from a RetroPie 4.5.1 on a RaspberryPi 3 and | ||
| # are included to make the image more realistic. One difference from | ||
| # base docker image is that .bashrc has colors enabled. | ||
| COPY docker/rpi/etc/skel /etc/skel |
There was a problem hiding this comment.
Known issue on Windows: When building a Linux image on Windows the container will always start with:
bash: $'\r': command not found
bash: /home/pi/.bashrc: line 6: syntax error near unexpected token `$'in\r''
'ash: /home/pi/.bashrc: line 6: `case $- inThe images built by Docker Hub work fine, so the issues below will be addressed when adding support for building on multiple architectures.
Issue 1
The line endings are \r\n on Windows, which gets copied into the image. There's a proposal from Nov '17 to give COPY/ADD better line ending support: docker/for-win#1340
Possible solutions now are:
- installing
dos2unixon the image and converting files during build - changing git
autocrlffor these files- I have already done this for
*.shfiles so I'll likely do this
- I have already done this for
Issue 2
File permissions change with COPY/ADD to include executable on Windows, as noted by the warning below. There's a proposal from Sep '17 to give COPY/ADD a chmod like flag: moby/moby#34819 (and a PR from Jul '19 moby/buildkit#1080)
SECURITY WARNING: You are building a Docker image from Windows against a non-Windows Docker host. All files and directories added to build context will have '-rwxr-xr-x' permissions. It is recommended to double check and reset permissions for sensitive files and directories.
Expected permissions on a RPi:
$ ls -la /etc/skel
-rw-r--r-- 1 root root 220 May 15 2017 .bash_logout
-rw-r--r-- 1 root root 3523 Apr 8 2019 .bashrc
-rw-r--r-- 1 root root 675 May 15 2017 .profileActual permissions on the Windows built image:
$ ls -la /etc/skel
-rwxr-xr-x 1 root root 220 Mar 19 07:07 .bash_logout
-rwxr-xr-x 1 root root 3523 Mar 19 07:07 .bashrc
-rwxr-xr-x 1 root root 675 Mar 19 07:08 .profileIt's wrong but I couldn't see any issue when running the container.
seriema
added a commit
that referenced
this pull request
Mar 26, 2020
Happens when mounting the VM share in mount-vm-share.sh. The issue has two parts: 1. "pi" does not have write access to `/dev/fuse`, not even with sudo, so mount-vm-share.sh fails to mount the drive. 2. `/dev/fuse` does not exist until runtime so it's not accessible when building the Docker image. The easiset solution is to give back root-membership to "pi". It was removed in the switch from `useradd` to `adduser` in PR #8 (67a9067). Not the same, but there might be something to learn from Docker's requirement on root access for mounting volumes: moby/moby#2259 Note: The "pi" user on a real RetroPie installation on a RPi is _not_ member of root. So this solution should be removed one day. Cleaned up Dockerfile. The instructions will be added to Readme.
seriema
added a commit
that referenced
this pull request
Mar 29, 2020
Revert "realistic bash user" (remove COPY layer in Dockerfile) `COPY` is used for "realistic bash user" (92f5129), but `COPY` comes with two main issues that made me decide to revert its use: Cache invalidation, and Windows issues. The former being the deciding factor, as the later is fixed. The long build times are not worth it. Since this `COPY` is for the user creation it's very early in the Dockerfile, and even with a fat cache layer before it (experimental in `docker-multiarch`) it still adds a 20min rebuild **locally** way too often. **Note:** This will revert the "realistic bash user" improvements introduced in PR #8, e.g. colored prompt, etc, that are found on a real RPi with RetroPie. It might still be achievable with RetroPie packages such as `bashwelcometweak`, and there are ways to make `COPY` more stable. So I might revisit "realistic bash user" at a later point.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Docker Hub takes 3 hours for a build so there are multiple slightly related changes in this one PR to consolidate the testing.
piis created. It differed from a real RetroPie installation on a Raspberry Pi 3 and usingadduseras the RPi docs show made it more similar. Included bash templates from a real RPi to make it even more similar, such as enabling color in the terminal.sshfs. It was speeding up the test runs as intended, but was hiding the fact thatraspberry-pi/mount-vm-share.shhad a user prompt that would cause the setup to hang in CI or a RPi, and the rest of the flow doesn't require the user to acceptapt-getinstalls.