Rethinking how runsc is built

[vrothberg]

Currently, the entire build system depends on Bazel. Although Bazel entails many advantages, it renders the project to be nearly impossible to build in certain environments. I have been working on creating a package for openSUSE since yesterday, tried different versions of Bazel to pre-fetch all dependencies into a cache, but all attempts failed. It seems there is no way to cache all dependencies into a separate directory, and re-use that in a different build.

The core problem is that the build environments of (nearly) all distributions do not allow to fetch dependencies via the network, which is an integral part of how Bazel works. Maybe we can find a way to satisfy both, using Bazel and making gvisor more distribution friendly?

[rogpeppe]

FWIW I tried to use bazel (installed with the instructions here) and it fails with this error:

% bazel build runsc
Couldn't find java at '/usr/lib/jvm/java-6-sun/bin/java'.

I think it would be great if at all possible that gvisor was buildable with the standard go tool.

At the least, it would be nice if source files could be grouped in the usual Go way (one command or package per directory). Example: it looks like there are two commands in the go_generics directory:

% go install
# gvisor.googlesource.com/gvisor/tools/go_generics
./merge.go:32:2: output redeclared in this block
	previous declaration at ./generics.go:106:2
./merge.go:40:6: main redeclared in this block
	previous declaration at ./generics.go:153:6
./merge.go:41:15: main.func1 redeclared in this block
	previous declaration at ./generics.go:154:15

[amscanne]

Thanks for the reports.

Re: java failure, it seems like your java installation might be broken. (Maybe an environment or alternatives issue?) I agree that the grouping should probably be fixed.

I'm afraid I'm not familiar with the OpenSUSE build environment. Is this build.opensuse.org? I'm looking into this now. I want to understand the issue fully before any drastic considerations.

[vrothberg]

I'm afraid I'm not familiar with the OpenSUSE build environment. Is this build.opensuse.org? I'm looking into this now. I want to understand the issue fully before any drastic considerations.

@amscanne That's correct. The core problem with bazel is that it fetches dependencies at build-time. I tried playing around with pre-fetching them using a cache and re-using that cache in the RPM build, but it doesn't seem to work as bazel will still miss some dependencies.

What would we optimal is to have all dependencies in ./vendor directory and to specify which libraries are additionally required at build-time and run-time. I was checking what other distros are doing with bazel and most have no packages, I assume because of the mentioned issue.

[rogpeppe]

Re: java failure, it seems like your java installation might be broken. (Maybe an environment or alternatives issue?) I agree that the grouping should probably be fixed.

FWIW I'm pretty sure I have a working java installation (I can run the "java" command fine), but no binary at /usr/lib/jvm/java-6-sun/bin/java (this is the listing of my /usr/lib/jvm directory).

It would be nice if I didn't need to debug java installation issues to try to build a Go tool.

[hugelgupf]

It would be nice if I didn't need to debug java installation issues to try to build a Go tool.

You're right, and the Go parts are definitely possible to build with vendoring. But keep in mind we also have the VDSO in C++ - which has to be built and included in the Go binary: https://github.com/google/gvisor/blob/master/vdso/BUILD

[iangudger]

There is also all of the generated Go code. The Go toolchain doesn't handle that very well. I believe that is part of the reason that Kubernetes is transitioning to Bazel.

[bluecmd]

@vrothberg @rogpeppe Have you raised these concerns with the Bazel team? It seems to me like bugs/valid feature requests, and fixing it there would help everyone that is using Bazel.

[vrothberg]

@vrothberg @rogpeppe Have you raised these concerns with the Bazel team? It seems to me like bugs/valid feature requests, and fixing it there would help everyone that is using Bazel.

@bluecmd, not yet. I first wanted to raise the issue here, which is likely to be a blocker for a wider adoption of gvisor on many Linux distributions. Some of Bazel's options just left the experimental stage with v0.13.0, so I assume that support for it is still at an early stage.

[davidstanke]

Hey folks! Bazel team here. Thanks for the feedback -- we'd love to understand your needs better and see how we might be able to help. I encourage you to raise issues in github.com/bazelbuild/bazel (or .../rules_go if that's appropriate), and I'd also be happy to hop on a call some time to talk through stuff. Just contact me at davidstanke [at] google [dot] com if you'd like to set up a chat.

[vrothberg]

Hi @davidstanke, thanks for checking! I opened an issue for Bazel (see bazelbuild/bazel#5175).

[...] and I'd also be happy to hop on a call some time to talk through stuff. Just contact me at davidstanke [at] google [dot] com if you'd like to set up a chat.

Thanks! Let's see how the discussion kicks-off in the Bazel issue. In case we cannot resolve the issue, setting up a call sounds great, thanks.

[fvoznika]

The issue with pre-fetching dependencies is being resolved in bazelbuild/bazel#5175.

Every build system has its pros and cons. I know that bazel is not widely adopted, but it fits pretty well with our project, considering we use it with the internal version of bazel.

[vrothberg]

Thanks for the info, @fvoznika!