Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI:DOCS] Document how to get secret mounts working on RHEL8 #18878

Merged

Conversation

imphil
Copy link
Contributor

@imphil imphil commented Jun 13, 2023

Add a section to the troubleshooting guide describing how to get secret mounts working in RHEL8, as discussed in
containers/buildah#4228.

It took me a while to find this information, and a RHEL8 default setup isn't that uncommon, so I thought it would be worth the change.

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jun 13, 2023

Adding the "do-not-merge/release-note-label-needed" label because no release-note block was detected, please follow our release note process to remove it.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci openshift-ci bot added the do-not-merge/release-note-label-needed Enforce release-note requirement, even if just None label Jun 13, 2023
@imphil imphil force-pushed the troubleshooting-secrets-crun branch 3 times, most recently from c81f5d4 to 99cb81e Compare June 13, 2023 16:48
@rhatdan rhatdan changed the title Document how to get secret mounts working on RHEL8 [CI:DOCS] Document how to get secret mounts working on RHEL8 Jun 13, 2023

### 41) A podman build step with `--mount=type=secret` fails with "operation not permitted"

Executing a step in a `Dockerfile`/`Containerfile` which mounts secrets using `--mount=type=secret` fails with "operation not permitted" when running on a host filesystem mounted with `nosuid` and when using the `runc` runtime (both of which are true by default on RHEL8).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Take out comments on RHEL8, they will not age well.


#### Solution

* Install `crun`, e.g. with `dnf install crun` on RHEL8.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove on RHEL8

#### Solution

* Install `crun`, e.g. with `dnf install crun` on RHEL8.
* Use the `crun` runtime by passing `--runtime /usr/bin/crun` to `podman buildx build` (or `podman build` with `DOCKER_BUILDKIT=1` set).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Podman build is the same podman buildx build. Also DOCKER_BUILDKIT=1 is ignored.
podman build always does the buildkit mode.

@imphil imphil force-pushed the troubleshooting-secrets-crun branch from 99cb81e to baa4529 Compare June 13, 2023 19:40
@imphil
Copy link
Contributor Author

imphil commented Jun 13, 2023

Thanks for your review, @rhatdan. Updated.

@rhatdan rhatdan removed the do-not-merge/release-note-label-needed Enforce release-note requirement, even if just None label Jun 13, 2023
@rhatdan
Copy link
Member

rhatdan commented Jun 13, 2023

/approve
LGTM
@giuseppe PTAL

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 13, 2023

#### Symptom

A `RUN` line in `Dockerfile`/`Containerfile` contains a [secret mount](https://github.com/containers/common/blob/main/docs/Containerfile.5.md) such as `--mount=type=secret,id=MY_USER,target=/etc/dnf/vars/MY_USER`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
A `RUN` line in `Dockerfile`/`Containerfile` contains a [secret mount](https://github.com/containers/common/blob/main/docs/Containerfile.5.md) such as `--mount=type=secret,id=MY_USER,target=/etc/dnf/vars/MY_USER`.
A `RUN` line in the `Dockerfile`/`Containerfile` contains a [secret mount](https://github.com/containers/common/blob/main/docs/Containerfile.5.md) such as `--mount=type=secret,id=MY_USER,target=/etc/dnf/vars/MY_USER`.

#### Symptom

A `RUN` line in `Dockerfile`/`Containerfile` contains a [secret mount](https://github.com/containers/common/blob/main/docs/Containerfile.5.md) such as `--mount=type=secret,id=MY_USER,target=/etc/dnf/vars/MY_USER`.
When running `podman buildx build` the process fails with an error message like
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

soft suggestion

Suggested change
When running `podman buildx build` the process fails with an error message like
When running `podman buildx build` the process fails with an error message like:

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove reference to buildx. Should just be podman build

* Install `crun`, e.g. with `dnf install crun`.
* Use the `crun` runtime by passing `--runtime /usr/bin/crun` to `podman build`.

See also https://github.com/containers/buildah/issues/4228 for a full discussion on the problem.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
See also https://github.com/containers/buildah/issues/4228 for a full discussion on the problem.
See also [Buildah issue 4228](https://github.com/containers/buildah/issues/4228) for a full discussion of the problem.

Add a section to the troubleshooting guide describing how to get secret
mounts working in an environment with nosuid-mounted file systems and
runc, such as RHEL8, as discussed in
containers/buildah#4228.

Signed-off-by: Philipp Wagner <phw@ibm.com>
@imphil imphil force-pushed the troubleshooting-secrets-crun branch from baa4529 to 0ecc0d9 Compare June 14, 2023 07:35
@imphil
Copy link
Contributor Author

imphil commented Jun 14, 2023

Thanks @TomSweeneyRedHat and @rhatdan, updated with your suggestions.

Copy link
Member

@giuseppe giuseppe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jun 14, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: giuseppe, imphil, rhatdan

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@TomSweeneyRedHat
Copy link
Member

LGTM
TYVM @imphil !

@TomSweeneyRedHat
Copy link
Member

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Jun 14, 2023
@openshift-merge-robot openshift-merge-robot merged commit 7db395a into containers:main Jun 14, 2023
@imphil imphil deleted the troubleshooting-secrets-crun branch June 14, 2023 22:14
@github-actions github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Sep 13, 2023
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 13, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants