Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

libcontainer: dmz: fix "go get" builds #4101

Merged
merged 1 commit into from
Nov 7, 2023
Merged

libcontainer: dmz: fix "go get" builds #4101

merged 1 commit into from
Nov 7, 2023

Conversation

cyphar
Copy link
Member

@cyphar cyphar commented Oct 30, 2023

Because runc-dmz is not checked into Git, go get will end up creating a copy of libcontainer/dmz with no runc-dmz binary, which causes external libcontainer users to have compilation errors.

Unfortunately, we cannot get go:embed to just ignore that there are no files matching the provided pattern, so instead we need to create a dummy file that matches the go:embed (which we check into Git and so go get will copy) and switch to embed.FS.

This is a little bit uglier, but at least it will fix external libcontainer users.

Fixes #4096
Signed-off-by: Aleksa Sarai cyphar@cyphar.com

@mythi
Copy link
Contributor

mythi commented Oct 30, 2023

FWIW, I tested this with my cAdvisor PR using

replace github.com/opencontainers/runc => github.com/cyphar/runc v1.0.0-rc4.0.20231030083055-39322bc53518

and without -tags runc_nodmz. Both make test and make build in cAdvisor built OK.

Copy link
Contributor

@kolyshkin kolyshkin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cyphar cyphar requested a review from kolyshkin November 2, 2023 01:34
@cyphar cyphar dismissed kolyshkin’s stale review November 2, 2023 01:34

switched to sync.Once

Copy link
Contributor

@kolyshkin kolyshkin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

I've also checked if it makes sense to get rid of debug/elf dependency in order to reduce runc binary size, as we only use a single constant from it.

Apparently not, probably because github.com/cilium/ebpf also uses it.

Comment on lines 21 to 23
// There is an empty file called runc-dmz-dummy-file.txt in libcontainer/dmz in
// order to work around the restriction that go:embed requires at least one
// file to match the pattern.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A nit. AFAIK we don't need runc-dmz-dummy-file.txt in libcontainer/dmz anymore since we have .gitignore in there (which, by the way, is also being embedded).

Or, we need to do

-//go:embed binary
+//go:embed binary/runc-*

so that .gitignore is not included.

Either way is fine.

Copy link
Member Author

@cyphar cyphar Nov 2, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually .gitignore is not embedded because files starting with _ and . are not included by go:embed when embedding a directory unless you use the all: prefix:

If a pattern names a directory, all files in the subtree rooted at that directory are embedded (recursively), except that files with names beginning with ‘.’ or ‘_’ are excluded. [...] If a pattern begins with the prefix ‘all:’, then the rule for walking directories is changed to include those files beginning with ‘.’ or ‘_’. For example, ‘all:image’ embeds both ‘image/.tempfile’ and ‘image/dir/.tempfile’.

If you remove runc-dmz and dummy-file.txt as well as the go:generate line in libcontainer/dmz/dmz_linux.go (to emulate using go get on runc), you'll find that the build fails with the same error as before.

@mythi
Copy link
Contributor

mythi commented Nov 2, 2023

github.com/cyphar/runc v1.0.0-rc4.0.20231102014201-9cc74bf7e259 built/tested in my env OK here too.

Because runc-dmz is not checked into Git, go get will end up creating a
copy of libcontainer/dmz with no runc-dmz binary, which causes external
libcontainer users to have compilation errors.

Unfortunately, we cannot get go:embed to just ignore that there are no
files matching the provided pattern, so instead we need to create a
dummy file that matches the go:embed (which we check into Git and so go
get _will_ copy) and switch to embed.FS.

This is a little bit uglier, but at least it will fix external
libcontainer users.

Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
// Verify that our embedded binary has a standard ELF header.
if !bytes.HasPrefix(runcDmzBinary, []byte(elf.ELFMAG)) {
if len(runcDmzBinary) != 0 {
logrus.Infof("misconfigured build: embedded runc-dmz binary is non-empty but is missing a proper ELF header")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be an error

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem is that we cannot tell if the embedded binary is broken at build-time, so making this an error would mean someone with an incredibly broken build system could result in a runc binary that fails to start any containers (when the alternative is we safely fallback to binary cloning).

I guess you could argue that at some point, we can't protect people from their own carelessness, but given that this can only be checked at runtime and falling back to /proc/self/exe cloning is completely safe, I feel like this is the right balance.

@AkihiroSuda AkihiroSuda mentioned this pull request Nov 6, 2023
21 tasks
@AkihiroSuda AkihiroSuda merged commit 5bcffdf into opencontainers:main Nov 7, 2023
45 checks passed
@cyphar cyphar deleted the runc-dmz-go-get-build branch November 7, 2023 02:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants