[bug] tls: bad certificate when gatekeeper cert gets rotated

[fseldow]

What happened in your environment?

When i manually rotate the gatekeeper cert, the ratify will not trust the new cert, reporting tls bad certificate. If i restart the ratify, everything will be working. It seems that we only load the cert when pod inits and will not update it.

Below is my steps:

1. Followed the quick start to install gatekeeper and ratify
2. Install constraint template and constraint, and it can pass the signed image
3. Mannually edit the secret gatekeeper-webhook-server-cert with a new cert and update the gatekeeper validating webhook configuration CA bundle. This step is to monitor the gatekeeper cert rotation
4. try kubectl run demo --image=wabbitnetworks.azurecr.io/test/notary-image:signed and reports error
   Error from server: admission webhook "mutation.gatekeeper.sh" denied the request: failed to resolve external data placeholders: failed to send external data request to provider ratify-mutation-provider: failed to send external data request: Post "https://ratify.gatekeeper-system:6001/ratify/gatekeeper/v1/mutate": remote error: tls: bad certificate
5. restart the ratify pod and re-run the sign image, everything becomes healthy

What did you expect to happen?

Since from ratify side, it cannot know when gatekeeper cert is updated. Maybe we should add mechanism to sync the cert for the server.
The question might also happen on ratify's cert.

What version of Kubernetes are you running?

1.26

What version of Ratify are you running?

v1.0.0-rc.3

Anything else you would like to add?

No response

Are you willing to submit PRs to contribute to this bug fix?

• Yes, I am willing to implement it.

[akashsinghal]

Thanks for pointing this out @fseldow. We will start discussions at next week's community call and aim to prioritize a fix.