feat: add configurable rekor address to workflows

[asraa]

Signed-off-by: Asra Ali asraa@google.com

Completes #372 to allow workflow inputs for go and generic.

[laurentsimon]

Awesome, thanks!

[asraa]

/hold doing some local testing (slow because of actions outage)

[asraa]

@laurentsimon need some advice :) because we're relying on a staging instance, we need to find the rekor public key of the instance out of band.

during upload in the generator we verify the tlog entry that was just uploaded. i could (insecurely imo) retrieve the pubkey to verify with FROM the instance itself by specifying an environment variable in the builders.

however, in the verifier, we need to document that if you're verifying with an alternative REKOR you need to specify an environment variable with the public key. wdyt?

[laurentsimon]

@laurentsimon need some advice :) because we're relying on a staging instance, we need to find the rekor public key of the instance out of band.

during upload in the generator we verify the tlog entry that was just uploaded. i could (insecurely imo) retrieve the pubkey to verify with FROM the instance itself by specifying an environment variable in the builders.

however, in the verifier, we need to document that if you're verifying with an alternative REKOR you need to specify an environment variable with the public key. wdyt?

Do you know what cosign plans to do for this? Do they ask users to install the root certs on their machine, out-of-band (and pick it up from a config file; or use the default PKI storage...)?

Does the staging instance of the official project have a known public key?

[asraa]

Do you know what cosign plans to do for this? Do they ask users to install the root certs on their machine, out-of-band (and pick it up from a config file; or use the default PKI storage...)?

Cosign expects that you EITHER initialize the custom TUF root with a mirror for the TUF repo, OR provide a public key file. The UX issue here is that the builder then requires setting a rekor-url AND your expected rekor public key (or we can only specify only allowing "staging/public instance" and I can grab the staging TUF repo)

Does the staging instance of the official project have a known public key?

Yeah! There is a staging TUF repo too. See above.

[laurentsimon]

Do you know what cosign plans to do for this? Do they ask users to install the root certs on their machine, out-of-band (and pick it up from a config file; or use the default PKI storage...)?

Cosign expects that you EITHER initialize the custom TUF root with a mirror for the TUF repo, OR provide a public key file. The UX issue here is that the builder then requires setting a rekor-url AND your expected rekor public key (or we can only specify only allowing "staging/public instance" and I can grab the staging TUF repo)

How about starting with the staging/public option? Can we hide it behind a VERIFIER_DEV_SOMETHING env variables too (or is that too much)? Hiding would let us break backward compatibility in the future once we have a better story around pubKey/config. Wdut?

Does the staging instance of the official project have a known public key?

Yeah! There is a staging TUF repo too. See above.

[asraa]

How about starting with the staging/public option? Can we hide it behind a VERIFIER_DEV_SOMETHING env variables too (or is that too much)? Hiding would let us break backward compatibility in the future once we have a better story around pubKey/config. Wdut?

That sounds good! Start simple. I'll revamp this PR then and expose a boolean to enable staging

[ianlewis]

How about starting with the staging/public option? Can we hide it behind a VERIFIER_DEV_SOMETHING env variables too (or is that too much)? Hiding would let us break backward compatibility in the future once we have a better story around pubKey/config. Wdut?

That sounds good! Start simple. I'll revamp this PR then and expose a boolean to enable staging

If I'm understanding correctly this PR is going to expose a boolean to allow us to use rekor staging for tests. And users should not use the option.

If that's right then SGTM. We can think a bit more about allowing private rekor servers and associated keys later.

[asraa]

Just finished doing a bit of debugging! Turns our // go:embed and //go:embed behave differently >.>

If I'm understanding correctly this PR is going to expose a boolean to allow us to use rekor staging for tests. And users should not use the option.

Yeah, this is mostly for our e2e tests. I can remove the README.md if I should keep this "hidden"

[asraa]

Successful run against staging: https://github.com/asraa/slsa-on-github-test/actions/runs/2598266989

[laurentsimon]

Just finished doing a bit of debugging! Turns our // go:embed and //go:embed behave differently >.>

How so?

If I'm understanding correctly this PR is going to expose a boolean to allow us to use rekor staging for tests. And users should not use the option.

Yeah, this is mostly for our e2e tests. I can remove the README.md if I should keep this "hidden"

yeah let's remove it and call the option internal-bla or experimental-bla?

[ianlewis]

Just finished doing a bit of debugging! Turns our // go:embed and //go:embed behave differently >.>

How so?

I think it's just that // go:embed (with a space) isn't picked up and is treated as just a regular comment.

Yeah, this is mostly for our e2e tests. I can remove the README.md if I should keep this "hidden"

yeah let's remove it and call the option internal-bla or experimental-bla?

Agreed, I think we will want to support properly in the future via a rekor address/port + public key input so we can support private rekor servers.

[asraa]

Agreed, I think we will want to support properly in the future via a rekor address/port + public key input so we can support private rekor servers.

All set.

Yeah, right now there's a complication that using private rekor servers also requires a root to trust that needs to be supplied out of band. for staging, sigstore manages one that i've used in this code. at that stage, we will need people to also supply their root as input. for another time.

I think it's just that // go:embed (with a space) isn't picked up and is treated as just a regular comment.

Yep :P

[laurentsimon]

BTW, I think in a PR on Sigstore, Bob commented that rekor staging instance is unstable. So can we reliably use it?

[asraa]

BTW, I think in a PR on Sigstore, Bob commented that rekor staging instance is unstable. So can we reliably use it?

Hmmm :/ I do know that there are pushes and updates to it frequently, so it is not using stable versions of releases. At this point I'm not sure what the guidance should be then. What should users who are testing be using then, other than prod? The pre-prod environment likely no as well, since that's for testing before pushing to prod. I will ask offline.

[ianlewis]

Is it just the rate limits we care about? Is one alternative to maybe set up our own server for e2e tests somewhere where we can set our own limits? (not that I care to maintain something like that)

[flxw]

Hi all, I recently stumbled over the fact that I wanted to make use of the container workflow with a private sigstore setup and couldn't configure the respective custom URLs and certificates.
I think the following workaround is acceptable and works fine for me. It may even eliminate the need for this PR:

jobs:
  provenance:
    needs: [build]
    permissions:
      actions: read # for detecting the Github Actions environment.
      id-token: write # for creating OIDC tokens for signing.
      packages: write # for uploading attestations.
    env:
      TUF_URL: https://my.custom.tuf.com
    steps:
      - name: Install cosign
        uses: sigstore/cosign-installer@7e0881f8fe90b25e305bbf0309761e9314607e25

      - name: Initialize cosign
        run: cosign initialize --root tuf-root.json --mirror $TUF_URL
        
      - uses: slsa-framework/slsa-github-generator/.github/workflows/generator_container_slsa3.yml@main
        with:
          image: ${{ needs.build.outputs.image }}
          digest: ${{ needs.build.outputs.digest }}
          registry-username: ${{ github.actor }}
          compile-generator: true
        secrets:
          registry-password: ${{ secrets.GITHUB_TOKEN }}

If a company wanted to use the workflow on a larger scale, they would simply need to bundle the steps above into a composable workflow, along with the public keys and URLs of the sigstore components. That would also be relatively low on maintenance.

[flxw]

Sorry, what I wrote doesn't make any sense. There still would need to be a way to customize rekor, fulcio and other URLs that are passed to cosign. So this PR remains valid, ofc. I don't really agree with restricting it to the staging sigstore, though.

[ianlewis]

Sorry, what I wrote doesn't make any sense. There still would need to be a way to customize rekor, fulcio and other URLs that are passed to cosign. So this PR remains valid, ofc. I don't really agree with restricting it to the staging sigstore, though.

I think this was meant as an interim solution until we figured out how to deal with setting the TUF root for private Rekor servers.

I think, this PR is moot though since we don't have the same rate limiting issues with the production server that we used to have.

[flxw]

Alright, if it's moot shall we close it and take the commits for using a private Rekor?
I'm also happy to do that :)

[asraa]

Alright, if it's moot shall we close it and take the commits for using a private Rekor?

I can close it!

I think we want both private Rekor and private Fulcio: e.g. the entire configured ecosystem