Proposal: Support for signatures via sigstore stack

[shibumi]

Summary

As of today (2021-11-29), krew does not support signature validation for itself nor for its plugins. Supporting signed release is non-trivial and GPG/OpenPGP is known to be difficult and error-prone. Although, downloads of krew and its plugins are secured by HTTPS on the transport way, this is not enough.

This proposal intends to provide signatures and signature validation via the sigstore stack. The following sigstore components are relevant for this issue:

• Rekor: A transparency log for all signatures. The public instance is located at https://rekor.sigstore.dev/api/v1/log/
• Cosign: A client and library for signing and validating images and binary large objects (BLOBs)
• Workload Identities via Github Actions

Advantages of the Sigstore stack:

• All signatures are being logged in a signature transparency log
• All signatures will be validated against the transparency log
• No private key handling necessary
• No trouble through maintainers "forgetting the PIN for their private key" or "losing their private key"
• Keyless signatures are easy to implement and difficult to do wrong (compared to GPG releases)

Architecture

The architecture is rather simple. In Krew we would use the cosign library to validate signed plugin releases via the rekor transparency log. Generating keyless signatures works as follows:

1. Creating a tag in the github repository creates a Github Action Workflow
2. The Github Action Workflow releases the plugin + signs the release via Github's OIDC (OpenID Connect) workload identity Issuer
3. The generated signatures will appear together with the released binaries/source tarballs on the release page

Examples

• The Kubernetes Release Team has announced SLSA 1 compliance and discusses cosign signed releases
• GoReleaser has full cosign support with version v1.0.0
• A full example for keyless signatures can be found here: https://shibumi.dev/posts/keyless-signatures-with-github-actions/

Milestones

1. Sign krew releases with cosign keyless
2. Provide a tutorial for sign krew plugins keyless
3. Implement cosign-backed signature checks for plugins
4. Grace Period? 3 months?
5. Enforce signature validation for future krew plugin installations

Long Term Goals

• Getting SLSA compliant: https://slsa.dev/levels

I am willing to contribute PRs, if I get the proper guidance and mentoring :)

[ahmetb]

Thanks for this proposal. I'll list several thoughts:

1. We currently have a post-install warning for all plugins (except for "krew" itself) that prints that this plugin is untrusted, as we (the krew maintainers) do not audit the source code of each plugin distributed via the centralized krew-index, and it is up to the user whether to trust what they install.

   Historically, we've done this to clarify this aspect of the plugin security model to the end-user (Add post-install disclaimer about plugin security #315, Add security notice to be shown after install and upgrade #316 and Print security notice when index is added #616). The benefits of these warnings can be debated again, but we've received no complaints so far either.

   Signing the plugins does not solve this issue for us, and these warnings are likely to remain in place as they were never about "trusting the build", but "trusting the author" in the first place.

2. Sigstore offers great features and I'm a big fan --but we need to nail down precisely how this benefits krew, as we already have a checksum specified for each version of a plugin, and authors are not allowed to modify release binaries without releasing a new version.

   (I suspect the answer here is something like "what if the author's GitHub account/access is compromised", and in that scenario I'm not sure how cosign helps, please illustrate what sort of attestations we'd look for in the cosign-produced signature.)

   What would be the visible benefit to the Krew user so that other plugin authors would want to start signing their releases? (Would we add a "verified" label somewhere, like Github does for commits?)

3. It's unlikely we would make signing mandatory (even in centralized krew-index) for the foreseeable future, as many of the plugins have been added a long time and not updated frequently. We need to continue those to work.

   Ease of new plugin onboarding is also something we try to make the experience as smooth as we can. If the hassle to understand and set up signing outweigh the benefits, we might end up not recommending it.

   For example, we ask every new plugin submission to use our GH action for making automated releases, and we can provide an optional tutorial to get started with signing, but it probably shouldn't be more than 5-10 minutes to get it working.

As we get through this, we can talk more about how the signatures would be stored in the releases and checked by krew. But for now, it's better if we work on answers to these first. I truly appreciate the effort here.

/cc @chriskim06
/cc @corneliusweig

[shibumi]

Hi Ahmet! Let me go through each of your points one by one:

Signing the plugins does not solve this issue for us, and these warnings are likely to remain in place as they were never about "trusting the build", but "trusting the author" in the first place.

I don't think this is so different to what Linux distributions do. Linux distributions are packaging foreign software and I honestly doubt that maintainers look into each commit of the packaged software. I am an Arch Linux maintainer myself, thus relying on the same trust you have to the plugin developers. The only difference between Linux distributions and krew might be, that Linux distributions tend to build the software on their own, while krew downloads pre-build binaries. How is brew doing this?

What would be the visible benefit to the Krew user so that other plugin authors would want to start signing their releases? (Would we add a "verified" label somewhere, like Github does for commits?)

Sigstore is not only about signing artifacts. I think the most interesting part about it is the transparency log. Yes, you have checksums and you download the binaries via HTTPS. This alone is already a good protection, especially in addition to HSTS, DNSSEC or other features the people might use anyway, but still I think it is not that different to Linux distributions.

As you know, supply chain security is much more than just signing releases. It is also about creating a software bill of materials and making each step in the chain verifiable and transparent.

However, while writing this I am missing a good answer to your question, too. To be honest, I have to admit that I have to back-off a little bit and "Proposal" might be the wrong wording for this issue. We might should see this as an open discussion and potential mind map for having a deeper look into the krew supply chain system (I might be a little bit rushy sometimes regarding new ideas :D ).

What would be the visible benefit to the Krew user so that other plugin authors would want to start signing their releases? (Would we add a "verified" label somewhere, like Github does for commits?)

Making it mandatory would only work over a very long grace period, I completely agree here with you.

For example, we ask every new plugin submission to use our GH action for making automated releases, and we can provide an optional tutorial to get started with signing, but it probably shouldn't be more than 5-10 minutes to get it working.

IMO, this might be even faster than just 5-10min. The sigstore stack makes signing very easy, especially with Goreleaser. I see that there are a few python or bash plugins as well.. signing those shouldn't be that complicated, too.

Some random thoughts

I spend some time on sketching the usual plugin release process:

This supply chain may be different depending on the maintainer, but it should match most plugins (I guess?!).

1. The maintainer commits new changes + sets a tag
2. The pipeline starts building the binaries
3. The pipeline creates a PR for the index OR the maintainer does this manually
4. An Index maintainer verifies the PR and approves it?? (Or is this being auto-approved? I have to admit I don't know the exact process behind this).

As you wrote earlier verifying if the release really comes from the Github Account is quite pointless, because the new release will only land in the index, if a PR for the plugin has been opened. Hence, if an attacker has access to the Github account he is able to create a malicious commit and release a new build.

The only benefit I see at the moment is the transparency log. With the transparency log it is possible to pin a tag to a specific Github action pipeline workflow and to a fixed datetime.

I think what we are actually looking for is end-to-end security in terms of: The maintainer signs a tag, the krew plugin sees the signed tag verifies it, adds it to the SBOM and starts the build.. pipeline then creates a PR and the krew index would auto-approve the new release if successfully verified.

@dlorenc do you see a benefit that I might not see? Every layer security makes it more difficult to tamper with the supply chain.
The transparency log might be the best argument for cosign support in krew, but that's it (except I am really missing something here)

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale