Support for provenance attestations #3240
Conversation
On Buildx, I wonder if we could do this automatically when build runs on CI and doesn't use a Git context so we can translate CI context to |
tonistiigi
force-pushed
the
attestations-provenance
branch
from
e3235c1 to
55a021f
Compare
| b, err := strconv.ParseBool(v) | ||
| if err != nil { | ||
| return nil, errors.Wrapf(err, "failed to parse reproducible flag %q", v) | ||
| } |
There was a problem hiding this comment.
Maybe warn if v is true, until we can support repro
There was a problem hiding this comment.
I think this would always be passed by the user. Even if network is disabled, a process can read something from hardware or just run a for loop and count how much time it takes. What we can detect automatically is that parameters and materials are complete and there is no network access(hermetic). These are all requirements for SLSA4 while the full byte-by-byte reproducibility is not.
There was a problem hiding this comment.
Left a few comments, mostly around the exporter/solver side, I'm not familiar enough with the cache changes to comment too strongly on them, though they look good at a glance.
There are a few conflicts with #3197, since they both add a Callback - not sure how you want to handle merge order for both of them.
tonistiigi
force-pushed
the
attestations-provenance
branch
3 times, most recently
from
536e5a2 to
fd4f899
Compare
| return res, bi, nil | ||
| } | ||
|
|
||
| func (b *llbBridge) Solve(ctx context.Context, req frontend.SolveRequest, sid string) (res *frontend.Result, err error) { |
There was a problem hiding this comment.
I think we should still implement Solve for llbBridge so that it continues to implement FrontendLLBBridge, unless there's a specific reason it's been left out?
The way I read the intention of the code: provenanceBridge is a wrapper for llbBridge that collects all of the relevant provenance as well as performing the standard Solve.
There was a problem hiding this comment.
llbBridge does not have a Solve anymore and we shouldn't just add a function if there is nothing that calls it. Maybe llbBridge is not the best name for it anymore? The LLB builds still happen through llbBridge but not through the Solve function signature but the loadResult function that is called by the ResultProxy.Result() and calls into to solve graph with proper cache config.
There was a problem hiding this comment.
Theoretically provenanceBridge can be also seen like a frontendBridge if that helps.
tonistiigi
force-pushed
the
attestations-provenance
branch
from
fd4f899 to
57553c2
Compare
|
CI green. |
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Change how provenance information is captured from builds. While previously frontend passed the buildinfo sources with metadata, now all information is captured through buildkit. A frontend does not need to implement buildinfo and can't set incorrect/incomplete buildinfo for a build result. All LLB operations can now collect as much provenance info as they like that will be used when making the attestation. Previously this was limited to a single Pin value. For example now we also detect secrets and SSH IDs that the build uses, or if it accesses network, if local sources are used etc.. The new design makes sure this can be easily extended in the future. Provenance capture can now detect builds that do multiple separate subsolves in sequence. For example, first subsolve gathers the sources for the build and second one builds from immutable sources without a network connection. If first solve does not participate in final build result it does not end up in provenance. Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
tonistiigi
force-pushed
the
attestations-provenance
branch
from
57553c2 to
fb917b9
Compare
|
The last push was just rebase with merge conflict fixes. Going to bring this in as don't want to rebase this very much. If any more comments from other maintainers then I can address them in follow-ups. I'll also follow-up with docs. |
| github.com/package-url/packageurl-go v0.1.0 | ||
| github.com/package-url/packageurl-go v0.1.1-0.20220428063043-89078438f170 |
There was a problem hiding this comment.
@tonistiigi "master" was required for this one? Did we ask the maintainer if they're open to tagging a new release?
| github.com/in-toto/in-toto-golang v0.3.3 | ||
| github.com/in-toto/in-toto-golang v0.3.4-0.20220709202702-fa494aaa0add |
There was a problem hiding this comment.
Saw that they tagged new releases since this PR, so opened #3415
This PR adds support for provenance attestations for any build. Provenance attestations describe for the build was performed and is eventually meant to replace the custom buildinfo structures in image config that we added in v0.10.
The format of the provenance follows SLSA specification https://slsa.dev/provenance/v0.2 .
There are two modes: default "min" mode and "max/full" mode. In Docker Buildx we expect to enable the min mode by default for all builds(replaces docker/buildx#1290).
The build parameters (build-args/labels) are only saved in the max mode. This is so to be on the safe side if any builds still use build-args for secret values. We may change this in the future.
In the max mode, provenance attestation will also list all the build steps with their LLB properties with a source map that maps them to the original source code(eg. to lines inside a Dockerfile). If an image is exported, then the build steps are also mapped to the specific layer that they modify.
When building from an immutable build context(eg. git repository), its value is mapped to
invocation.configSource. When building from a local directory that BuildKit can't verify, the client can pass additionalvcs:arguments to provide metadata linking the build to a Git repository. All the sources used by the build in LLB level are added to the materials array, pinned with their immutable digests.You can see an example of full provenance attestation in https://gist.github.com/tonistiigi/23b3770f15422f5cce4bcd73773ab871#file-provenance-json
Current implementation is based on the information gathered by the previous buildinfo implementation. There are some issues with that implementation where not all correct fields are captured in all cases. I'm working on a complete refactoring of the buildinfo code to fix these issues.
These updates should be in a follow-up PR in the same milestone.Edit: This has been completed.