relation with GoogleContainerTools/distroless #48
Comments
|
@MarkusTeufelberger @adragomir Would either of you like to comment on that overlap in downloading packages? My efforts right now are in the package building side. Specifically I want to get that out of Bazel core and let the community run with this. Having package consumption rules in the same repo as package building rules might be an aid to development - but that is not a foregone conclusion. The rules_docker people might have some thoughts on their preferred deb handling. |
|
Friendly ping. |
|
The problem with distroless is that they are explicitly not considering security a feature and are just randomly trusting a non-debian mirror server without validating signatures. Their design also doesn't allow this to easily change and the last time I tried to work with them there, they didn't even acknowledge that this could be a problem. If anything, I'd like to see distroless remove their insecure implementation and start validating hashes and signatures. I'd argue that custom rules for pulling packages are also not at the ideal place for a project that aims to provide docker images with a few select binaries. I wouldn't be opposed to a different repository for package pull rules though, in case this is too confusing for people (rules_pkg_build vs. rules_pkg_pull?). There is a clear overlap here, since the code stems from an attempt by me to rewrite the distroless rules in a secure way. I'd also like to see convergence, but I don't know how the situation at distroless is these days and if they would be willing to accept patches that improve their security settings. There would be a few easy wins, but in the end they would need quite a bit of redesign to be able to check signatures reliably or define actually which version of which package gets installed from which version of Debian/Ubuntu/Mint/???. Personally I have to admit though that I kinda gave up on Bazel for now, as features I would have liked to see (bootstrapable builds [a means to compile and use your own TOOLCHAIN so things like in here or distroless where one needs to pull binaries from Google servers even if the repository contains all the code and the build process for these binaries], a collection of BUILD files maaaybe a bit like buckaroo.pm...) still haven't manifested and remote building as well as sandboxing seems still as clunky as ever. Maybe in a few years I'll take another look. If there is something concrete and I get the OK from people here and distroless who actually have commit access I would take another look, for now I just try to stay away from these projects as I just get a bit frustrated everytime I see something like |
|
Thank you for that informative and frank answer. FWIW. I expect to see some users start a community based effort to gather real world requirements for WORKSPACE dependency management. The concerns they already raised were how to verify packages, how to override external deps, how to properly pre-sync a client so you can build without downloads on the fly, how to bless what is installed on the system rather than pull it from the internet, and how to tell what versions of what packages are used in any target. They are mostly from the rpm and maven camps, but it is easy to see the concerns completely overlap with debian package reuse. My admission to them was that users are better able to determine the requirements for workspace dependency management than Google. We have a set of source control practices which avoid important real-world use cases, so our needs are different. |
|
Originally the whole rules_pkg repo was actually intended to be a collection for BUILD.bazel files for popular projects (something that Google probably has internally next to their vendored external dependencies) by the way - so kinda building really concrete packages of software/libraries as a drop-in. Maybe similar to https://github.com/nelhage/rules_boost as an example for a single library or crates.io as an example for how this could be presented to end users. That was never really gaining traction though so far since there's no need for this inside Google and outside Google a lot of larger projects seem to either drop Bazel, support a second build system anyways or have their own tooling (like gazelle) around generating BUILD files on the fly instead of relying on a central project. Also after a while of looking through build scripts of popular projects it starts to really get to you how a computer is even able to build or run anything... shell scripts, autotools, python/perl/someone's-home-cooked-DSL build scripts, generated code, "only builds with X compiler version", missing includes, code only available in a tarball every second full moon when the dev turns on their home router... Still nice to fantasize about every once in a while to "just" build a full distro in one run from source, but damn - there's a reason that there were probably whole teams at Google working on this for a few years in the past. |
|
Hi Markus, Thanks for the nice reply. That's really informative. Just would like to drop my cent(s?) on the lack of security of distroless: My guess is that maybe google maintains its internal mirrors that they trust so lack of securities in distroless is a non-concern to them. I have been using Bazel since |
|
I don't see a situation where there's overlap. Instead I see a gap. There's a public need for this functionality, but neither Distroless nor rules_pkg seems to want to offer rules that are consumable by the public. To me it makes more sense that deb repository rules belong with rules_pkg. Additionally, the rules here look to be better suited for general consumption. |
|
More and more I am moving the other way. I think the entire concept of how
we use packages in a WORKSPACE should be reevaluated. All rules that bring
things in need better coordination (for a variety of provenance and license
compliance issues) and should be combined into a single workspace with many
contributors. I would have consumers of debian, cmake, configure, and
others all in a single repository adjunct to Bazel. Perhaps rules_workspace.
…On Wed, Aug 5, 2020 at 10:35 PM Marius Grigoriu ***@***.***> wrote:
I don't see a situation where there's overlap. Instead I see a gap.
There's a public need for this functionality, but neither Distroless nor
rules_pkg seems to want to offer rules that are consumable by the public.
To me it makes more sense that deb repository rules belong with rules_pkg.
Additionally, the rules here look to be better suited for general
consumption.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#48 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAXHHHFDK2T7MYVKFPZT35DR7IJGHANCNFSM4H5CDPEQ>
.
|
|
I can respect that you don't want this project to turn into a junk drawer. Perhaps we should spin off the deb_packages into |
|
That would be fine. But it needs owners to step up and create it. The most
likely path to that happening is to create the rules in some other github
org (not bazelbuild) so that it can move at its own speed.
Moving forward, we would like more recommended rule sets to be owned by
others, instead of bazelbuild. We don't have to be the gatekeeper.
…On Tue, Aug 11, 2020 at 3:13 AM Marius Grigoriu ***@***.***> wrote:
I can respect that you don't want this project to turn into a junk drawer.
Perhaps we should spin off the deb_packages into rules_dpkg.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#48 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAXHHHHAD4DZSYVSUESZRVLSADVQVANCNFSM4H5CDPEQ>
.
|
|
Looks like I'm the one stepping up. I created a new repository with a copy of the |
I noticed that the
deb_packagescontains rules of similar functionalities with what's provided in thedistroless/package_managerdirectory:https://github.com/GoogleContainerTools/distroless/tree/master/package_manager
I'd like to see a convergence between these two efforts, given how much they overlap.
I understand that they are different for different reasons. Just would like to get onto the same page with everyone.
The text was updated successfully, but these errors were encountered: