From 77cac456ab23b78613e9c19731a094d4bb07e322 Mon Sep 17 00:00:00 2001 From: Qingyang Chen Date: Thu, 19 Apr 2018 17:23:01 -0400 Subject: [PATCH 1/8] Adds directory for proposals. --- proposals/README.md | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) create mode 100644 proposals/README.md diff --git a/proposals/README.md b/proposals/README.md new file mode 100644 index 0000000000..4e4af30276 --- /dev/null +++ b/proposals/README.md @@ -0,0 +1,21 @@ +# Design Documents and Proposals + +This directory holds the approved design documents and proposals for changes and new features to Jib. + +## Contribute + +### Submit a Proposal + +Submit a proposal by filing a pull request that adds a `.md` file to this directory. + +Follow the general layout of existing proposals. In general, make sure to include: + +- Description of the problem +- Goals to achieve +- Proposed solution to the problem + +### Review a Proposal + +Review pending proposals by commenting on [their pull requests](/../../issues?q=is%3Aissue+is%3Aopen+label%3Aproposal). Proposals in review are labelled with `proposal`. + +For approved proposals, you may open revision pull requests with your suggestions for revision, or provide your comments in a new [issue](/../../issues/new?body=) or messaging the [community](/../../#community). From 277beee450696cff8a3f0c0cb2a2ef045444b41e Mon Sep 17 00:00:00 2001 From: Qingyang Chen Date: Thu, 19 Apr 2018 17:25:38 -0400 Subject: [PATCH 2/8] Fixes link. --- proposals/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/README.md b/proposals/README.md index 4e4af30276..ef387eeb98 100644 --- a/proposals/README.md +++ b/proposals/README.md @@ -16,6 +16,6 @@ Follow the general layout of existing proposals. In general, make sure to includ ### Review a Proposal -Review pending proposals by commenting on [their pull requests](/../../issues?q=is%3Aissue+is%3Aopen+label%3Aproposal). Proposals in review are labelled with `proposal`. +Review pending proposals by commenting on [their pull requests](/../../pulls?q=is%3Aopen+is%3Apr+label%3Aproposal). Proposals in review are labelled with `proposal`. For approved proposals, you may open revision pull requests with your suggestions for revision, or provide your comments in a new [issue](/../../issues/new?body=) or messaging the [community](/../../#community). From fc5bca7b78bd337daaab71ea4dee81079b25fc4d Mon Sep 17 00:00:00 2001 From: Qingyang Chen Date: Thu, 19 Apr 2018 17:26:13 -0400 Subject: [PATCH 3/8] Fixes link. --- proposals/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/README.md b/proposals/README.md index ef387eeb98..efe7a792bb 100644 --- a/proposals/README.md +++ b/proposals/README.md @@ -18,4 +18,4 @@ Follow the general layout of existing proposals. In general, make sure to includ Review pending proposals by commenting on [their pull requests](/../../pulls?q=is%3Aopen+is%3Apr+label%3Aproposal). Proposals in review are labelled with `proposal`. -For approved proposals, you may open revision pull requests with your suggestions for revision, or provide your comments in a new [issue](/../../issues/new?body=) or messaging the [community](/../../#community). +For approved proposals, you may open revision pull requests with your suggestions for revision, or provide your comments in a new [issue](/../../issues/new?body="") or messaging the [community](/../../#community). From ac29a3f818d73aa5f6fc2d0fc3b69e33c5c59277 Mon Sep 17 00:00:00 2001 From: Qingyang Chen Date: Thu, 19 Apr 2018 17:27:41 -0400 Subject: [PATCH 4/8] Fixes link. --- proposals/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/README.md b/proposals/README.md index efe7a792bb..152e657384 100644 --- a/proposals/README.md +++ b/proposals/README.md @@ -18,4 +18,4 @@ Follow the general layout of existing proposals. In general, make sure to includ Review pending proposals by commenting on [their pull requests](/../../pulls?q=is%3Aopen+is%3Apr+label%3Aproposal). Proposals in review are labelled with `proposal`. -For approved proposals, you may open revision pull requests with your suggestions for revision, or provide your comments in a new [issue](/../../issues/new?body="") or messaging the [community](/../../#community). +For approved proposals, you may open revision pull requests with your suggestions for revision, or provide your comments in a new [issue](/../../issues/new?body=<!-- Please provide the link to the approved proposal you are commenting on. -->) or messaging the [community](/../../#community). From 0c1c4a04b02c79d5f07f995e2d793d731501a123 Mon Sep 17 00:00:00 2001 From: Qingyang Chen Date: Thu, 19 Apr 2018 17:28:39 -0400 Subject: [PATCH 5/8] Fixes link. --- proposals/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/README.md b/proposals/README.md index 152e657384..a2d4e5235d 100644 --- a/proposals/README.md +++ b/proposals/README.md @@ -18,4 +18,4 @@ Follow the general layout of existing proposals. In general, make sure to includ Review pending proposals by commenting on [their pull requests](/../../pulls?q=is%3Aopen+is%3Apr+label%3Aproposal). Proposals in review are labelled with `proposal`. -For approved proposals, you may open revision pull requests with your suggestions for revision, or provide your comments in a new [issue](/../../issues/new?body=<!-- Please provide the link to the approved proposal you are commenting on. -->) or messaging the [community](/../../#community). +For approved proposals, you may open revision pull requests with your suggestions for revision, or provide your comments in a new [issue](/../../issues/new?body=<!--%20Please%20provide%20the%20link%20to%20the%20approved%20proposal%20you%20are%20commenting%20on.%20-->) or messaging the [community](/../../#community). From 5e0b943efd901930ac5824c3850b548229961e8e Mon Sep 17 00:00:00 2001 From: Qingyang Chen Date: Fri, 20 Apr 2018 12:21:27 -0400 Subject: [PATCH 6/8] Adds clause about approval. --- proposals/README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/proposals/README.md b/proposals/README.md index a2d4e5235d..bb7ea6c8f0 100644 --- a/proposals/README.md +++ b/proposals/README.md @@ -14,6 +14,8 @@ Follow the general layout of existing proposals. In general, make sure to includ - Goals to achieve - Proposed solution to the problem +Proposals are approved upon merge. + ### Review a Proposal Review pending proposals by commenting on [their pull requests](/../../pulls?q=is%3Aopen+is%3Apr+label%3Aproposal). Proposals in review are labelled with `proposal`. From 7e6f2b2cd3aaa6cf4f3c97d044a990bbcacec580 Mon Sep 17 00:00:00 2001 From: Qingyang Chen Date: Fri, 20 Apr 2018 15:47:25 -0400 Subject: [PATCH 7/8] Adds proposal for addressing base image reproducibility. --- proposals/reproducible_base_image.md | 50 ++++++++++++++++++++++++++++ 1 file changed, 50 insertions(+) create mode 100644 proposals/reproducible_base_image.md diff --git a/proposals/reproducible_base_image.md b/proposals/reproducible_base_image.md new file mode 100644 index 0000000000..774a8f5ef1 --- /dev/null +++ b/proposals/reproducible_base_image.md @@ -0,0 +1,50 @@ +# Proposal: Address Base Image Reproducibility + +**Status:** *In Review* + +## Motivation + +One of the main goals of Jib is to be able to build images reproducibly, such that the same contents always creates the same images. It does this by wiping the timestamps and user information from the files in the Java application layers (dependencies, resources, classes). However, it does not do the same for the base image layers, which, by default, are from the latest [`gcr.io/distroless/java`](gcr.io/distroless/java) image. This may be unexpected behaviors since by default, reproducibility is on (the user may switch it off using the `enableReproducibleBuilds` parameter for Maven or the `reproducible` parameter for Gradle). + +### Terminology + +Image **reference** refers to the full reference for an image. This can be as short as `busybox` (which refers to the `library/busybox` **repository** on the Docker Hub **registry**), or as long as `gcr.io/distroless/java@sha256:0135c8b1adb3ed906f521973f825cea3fcdcb9b0db2f4012cc05480bf4d53fd6` (which refers to the image with **digest** `sha256:0135c8b1adb3...` in the `distroless/java` repository on the `gcr.io` registry). An image reference without a specific digest or tag, like `gcr.io/distroless/java`, defaults to the `latest` **tag**, which always refers to the newest digest in that repository. + +## Problem + +The main problem is that the reproducibility feature of Jib does not actually guaranteed *for the image*, but rather only guarantees reproducibility *for the application layers*. This is a bug. + +The problem arises in a common workflow where the developer expects reproducibility: + +1. The developer commits a change as version 123. +1. The developer builds the image for that commit - results in image A. +1. On another machine (possibly in prod), that developer checks out version 123 and builds the image - this should have resulted in image A again. + +However, since Jib uses the latest version of the [gcr.io/distroless/java](gcr.io/distroless/java) image (which is updated rather frequently - about every 2 weeks) as the base image to build the application layers on top of, if a newer [gcr.io/distroless/java](gcr.io/distroless/java) is latest, the rebuild would result in a different image than expected. + +## Goals + +- Maintain ease-of-use (no unnecessary extra configuration, at least for the default case) +- Preferable: Keep reproducibility on by default + +## Solution + +Jib will still use `gcr.io/distroless/java` by default, since in development, users may wish to keep at the latest base image. An alternative would be to use a specific digest of `gcr.io/distroless/java` but that would involve tying a version of Jib to a version of distroless. + +The `reproducible`/`enableReproducibleBuilds` configuration will be removed. Application layers (dependencies, resources, classes) will always be reproducible. + +Reproducibility will be guaranteed if the user specifies a specific digest to use for a base image. This can be specified as a fully-qualified custom base image, or as a `tag` configuration (Maven). + +The user will be warned if the base image used is tagged with `latest` such that reproducibility is not guaranteed. Note that this warning is given by default. + +So, the logic flow would be: + +1. Jib uses `gcr.io/distroless/java` as the base image. +1. If the user specifies a different image to use as the base image, use that. +1. The user can configure a specific digest to use - `tag` for Maven, and `from.image` for Gradle. +1. If the final tag/digest is still `latest`, warn the user that reproducibility is not guaranteed due a changeable base image, and suggest the user to specify a specific digest. + +## Implementation + +- Remove the `reproducible`/`enableReproducibleBuilds` configuration and always build application layers reproducibly. +- When validating the `jib-maven-plugin`/`jib-gradle-plugin` configuration, warn the user if the base image uses a `latest` tag. From ed5a3064a86aac89593d6a6bdb6a70683fe13757 Mon Sep 17 00:00:00 2001 From: Qingyang Chen Date: Wed, 25 Apr 2018 11:16:00 -0400 Subject: [PATCH 8/8] Removes status. --- proposals/reproducible_base_image.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/proposals/reproducible_base_image.md b/proposals/reproducible_base_image.md index 774a8f5ef1..5e9d8c32a9 100644 --- a/proposals/reproducible_base_image.md +++ b/proposals/reproducible_base_image.md @@ -1,7 +1,5 @@ # Proposal: Address Base Image Reproducibility -**Status:** *In Review* - ## Motivation One of the main goals of Jib is to be able to build images reproducibly, such that the same contents always creates the same images. It does this by wiping the timestamps and user information from the files in the Java application layers (dependencies, resources, classes). However, it does not do the same for the base image layers, which, by default, are from the latest [`gcr.io/distroless/java`](gcr.io/distroless/java) image. This may be unexpected behaviors since by default, reproducibility is on (the user may switch it off using the `enableReproducibleBuilds` parameter for Maven or the `reproducible` parameter for Gradle).