Trust-manager integration #7532

pierDipi · 2024-01-05T16:16:05Z

Add support for trust-manager.

This will allow administrators to add trusted CA bundles to Eventing components by creating ConfigMaps labeled networking.knative.dev/trust-bundle=true in the knative-eventing namespace.

For now, the rotation is not yet implemented, for that we would need to re-create clients when trusted bundles change

TODO:

bundle certs rotation in in-use clients

Fixes #

Proposed Changes

Trust-manager integration
Allow administrators to add CA trust bundle to all Eventing components

Pre-review Checklist

At least 80% unit test coverage
E2E tests for any new behavior
Docs PR for any user-facing impact
Spec PR for any new API feature
Conformance test for any change to the spec

Release Note

Add trust-manager integration, Knative Eventing components will add to the trusted Certificate Authorities (CAs) any PEM-encoded CA certificates in any `ConfigMap` in the `knative-eventing` namespace labeled with `networking.knative.dev/trust-bundle=true`

Docs

Issue Documenent Eventing trust-manager integration docs#5815

knative-prow · 2024-01-05T16:16:10Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: pierDipi

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [pierDipi]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

codecov · 2024-01-08T08:55:22Z

Codecov Report

Attention: 236 lines in your changes are missing coverage. Please review.

Comparison is base (36b3115) 75.57% compared to head (b1bc237) 74.54%.
Report is 2 commits behind head on main.

Files	Patch %	Lines
pkg/eventingtls/trust_bundle.go	1.87%	156 Missing and 1 partial ⚠️
pkg/adapter/v2/main.go	19.35%	24 Missing and 1 partial ⚠️
pkg/eventingtls/eventingtls.go	29.62%	18 Missing and 1 partial ⚠️
pkg/apis/sources/v1/sinkbinding_lifecycle.go	87.50%	5 Missing and 3 partials ⚠️
cmd/webhook/main.go	0.00%	6 Missing ⚠️
pkg/reconciler/apiserversource/apiserversource.go	62.50%	4 Missing and 2 partials ⚠️
pkg/reconciler/containersource/containersource.go	57.14%	2 Missing and 1 partial ⚠️
pkg/broker/filter/filter_handler.go	75.00%	2 Missing ⚠️
pkg/broker/ingress/ingress_handler.go	75.00%	2 Missing ⚠️
pkg/kncloudevents/http_client.go	81.81%	2 Missing ⚠️
... and 3 more

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #7532      +/-   ##
==========================================
- Coverage   75.57%   74.54%   -1.03%     
==========================================
  Files         261      262       +1     
  Lines       14668    14954     +286     
==========================================
+ Hits        11085    11148      +63     
- Misses       3004     3218     +214     
- Partials      579      588       +9

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

…figmaps Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

pkg/reconciler/containersource/containersource.go

pkg/reconciler/sinkbinding/sinkbinding.go

third_party/cert-manager/00-namespace.yaml

…in SB reconciler Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

pierDipi · 2024-01-10T10:21:49Z

/test conformance-tests

pierDipi · 2024-01-10T11:04:19Z

@creydr can you please take a look again?

creydr

Looking good. Only one nit. Thanks for addressing this!
/lgtm

/hold
Holding in case you want more reviews or want to address the nit. Feel free to unhold otherwise.

pkg/eventingtls/trust_bundle.go

pierDipi · 2024-01-10T13:32:42Z

@creydr I will address the nit in the follow up PR when implementing "bundle certs rotation in in-use clients"

/unhold

This is the data plane only part related to the trust-manager integration: knative/eventing#7532 Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

* Trust-manager integration Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Add E2E tests Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Fix linter and format Go code Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Upgrade rekt Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Reference Eventing CA issuer in E2E tests Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Force GVK in sources when propagating trust bundle Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Format Go code Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * SinkBinding reconciler needs to create/update/delete trust bundle configmaps Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Upgrade rekt deps Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Add Deployment volumes for SinkBinding tests Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Fix CA certs bundle and https endpoint assertions Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Pass TrustBundleConfigMapLister in PingSource runner Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Fix adapter create informer logic for trust bundle ConfigMaps Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Remove propagate configmaps in ContainerSource reconciler as is done in SB reconciler Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Inject trust bundles as volumes in sinkbinding reconciler Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Inject trust-bundle to SinkBinding subjects Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Fix lister ctx injection Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> --------- Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

* Label OpenShift CA trust bundle as Knative trust bundle Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Trust-manager integration (knative#7532) * Trust-manager integration Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Add E2E tests Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Fix linter and format Go code Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Upgrade rekt Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Reference Eventing CA issuer in E2E tests Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Force GVK in sources when propagating trust bundle Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Format Go code Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * SinkBinding reconciler needs to create/update/delete trust bundle configmaps Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Upgrade rekt deps Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Add Deployment volumes for SinkBinding tests Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Fix CA certs bundle and https endpoint assertions Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Pass TrustBundleConfigMapLister in PingSource runner Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Fix adapter create informer logic for trust bundle ConfigMaps Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Remove propagate configmaps in ContainerSource reconciler as is done in SB reconciler Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Inject trust bundles as volumes in sinkbinding reconciler Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Inject trust-bundle to SinkBinding subjects Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Fix lister ctx injection Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> --------- Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Reload trust-bundle on new connections (knative#7567) This allows administrators to dynamically add trust-bundles that are picked by clients when connecting to new hosts. Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Actually run TLS tests Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Run generate-release Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Create trust bundle ConfigMap manually Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Assign global resync Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Fix regex Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * User-provided issuers and CA certificate Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Fix configmap -> secret Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Label bundle configmap Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Base64 decode CA certs Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Remove install script Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Use projected volumes Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Create bundle on install_serverless so it's always done Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> --------- Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

…nsions#3594) This is the data plane only part related to the trust-manager integration: knative/eventing#7532 Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

* Support adding trust bundles to data place HTTP clients (knative-extensions#3594) This is the data plane only part related to the trust-manager integration: knative/eventing#7532 Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Propagate trust bundles in the data plane contract (knative-extensions#3605) * Propagate trust bundles in the data plane contract This is the follow up to knative-extensions#3594 for the control plane reconcilers. - Refactor prober's clients to use `DialTLSContext` to dynamically pass TLS config based on the current values of the trust bundles + CACerts field Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Add E2E tests Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Add consumer group and consumer CA certs status propagation Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Add comment to clarify PEM trust bundle validation Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Format code Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Set subscriberCACerts when scaled to 0 Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Add CA certs fields to KafkaChannel CRD Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Pass CA certs to contract for channel Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> --------- Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Skip KafkaSource TLS tests on Istio (knative-extensions#3633) * Skip KafkaSource TLS tests on Istio * Run goimport * Use midstream as eventing dependency Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Use new issuer Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Add configuration for E2E tests Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Use Class instead of record Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> --------- Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> Co-authored-by: Christoph Stäbler <cstabler@redhat.com>

Leo6Leo · 2024-01-30T19:02:51Z

hack/update-cert-manager.sh

+  helm repo add jetstack https://charts.jetstack.io --force-update
+  kubectl create namespace --dry-run=client cert-manager -oyaml > third_party/cert-manager/00-namespace.yaml
+  helm template -n cert-manager cert-manager jetstack/cert-manager --create-namespace --version "${cert_manager_version}" --set installCRDs=true > third_party/cert-manager/01-cert-manager.yaml
+  helm template -n cert-manager cert-manager jetstack/trust-manager --create-namespace --version "${trust_manager_version}" --set installCRDs=true > third_party/cert-manager/02-trust-manager.yaml


helm with version earlier than v3.13 has some bugs, as it cannot detect the right kubernetes version.

Error: chart requires kubeVersion: >= 1.22.0-0 which is incompatible with Kubernetes v1.20.0

@pierDipi

@Leo6Leo where did you see it? In CI we don't have this problem (as we use always the latest helm 3 version): https://github.com/knative-extensions/knobots/actions/runs/7719799980/job/21043681810
When it's locally, can up update?

helm template doesn't run against a cluster, it's just rendering the template

@creydr It has the problem locally. Added the requirement to the development guide in this #7643

knative-prow bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 5, 2024

knative-prow bot added approved Indicates a PR has been approved by an approver from all required OWNERS files. area/test-and-release Test infrastructure, tests or release size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Jan 5, 2024

knative-prow bot requested review from evankanderson, lberk and mgencur January 5, 2024 16:16

Trust-manager integration

f69b0fa

Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

pierDipi force-pushed the trust-manager branch from a420355 to f69b0fa Compare January 8, 2024 08:49

pierDipi mentioned this pull request Jan 8, 2024

Allow testing trust-bundle integration knative-extensions/reconciler-test#645

Merged

pierDipi added 9 commits January 8, 2024 11:14

Add E2E tests

338aab6

Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

Fix linter and format Go code

0c36def

Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

Upgrade rekt

1a4c290

Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

Reference Eventing CA issuer in E2E tests

177a1f3

Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

Force GVK in sources when propagating trust bundle

2363ab9

Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

Format Go code

f7809ee

Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

SinkBinding reconciler needs to create/update/delete trust bundle con…

ef45602

…figmaps Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

Upgrade rekt deps

7b010a7

Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

Add Deployment volumes for SinkBinding tests

fe3445e

Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

knative-prow-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 8, 2024

Merge branch 'main' into trust-manager

1430615

knative-prow-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 8, 2024

Fix CA certs bundle and https endpoint assertions

e2f7aad

Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

pierDipi mentioned this pull request Jan 8, 2024

Documenent Eventing trust-manager integration knative/docs#5815

Closed

pierDipi changed the title ~~[WIP] Trust-manager integration~~ Trust-manager integration Jan 8, 2024

knative-prow bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 8, 2024

pierDipi requested review from matzew and creydr January 8, 2024 16:17

pierDipi mentioned this pull request Jan 9, 2024

Eventing TLS: Support exposing cluster-local CA Certs for addressables resources knative/serving#14196

Closed

creydr reviewed Jan 9, 2024

View reviewed changes

pkg/reconciler/containersource/containersource.go Outdated Show resolved Hide resolved

pkg/reconciler/sinkbinding/sinkbinding.go Outdated Show resolved Hide resolved

third_party/cert-manager/00-namespace.yaml Show resolved Hide resolved

pierDipi added 3 commits January 9, 2024 15:45

Remove propagate configmaps in ContainerSource reconciler as is done …

1212ef1

…in SB reconciler Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

Inject trust bundles as volumes in sinkbinding reconciler

8f93dd1

Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

Inject trust-bundle to SinkBinding subjects

f7e5f7d

Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

pierDipi requested a review from creydr January 9, 2024 17:40

Fix lister ctx injection

b1bc237

Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

pierDipi force-pushed the trust-manager branch from 23034ad to b1bc237 Compare January 10, 2024 09:50

creydr reviewed Jan 10, 2024

View reviewed changes

pkg/eventingtls/trust_bundle.go Show resolved Hide resolved

knative-prow bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 10, 2024

knative-prow bot assigned creydr Jan 10, 2024

knative-prow bot added the lgtm Indicates that a PR is ready to be merged. label Jan 10, 2024

knative-prow bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 10, 2024

knative-prow bot merged commit a7166fc into knative:main Jan 10, 2024
36 of 41 checks passed

pierDipi deleted the trust-manager branch January 10, 2024 13:35

pierDipi mentioned this pull request Jan 11, 2024

Support adding trust bundles to data place HTTP clients knative-extensions/eventing-kafka-broker#3594

Merged

Leo6Leo reviewed Jan 30, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Trust-manager integration #7532

Trust-manager integration #7532

pierDipi commented Jan 5, 2024 •

edited

Loading

knative-prow bot commented Jan 5, 2024

codecov bot commented Jan 8, 2024 •

edited

Loading

pierDipi commented Jan 10, 2024

pierDipi commented Jan 10, 2024

creydr left a comment

pierDipi commented Jan 10, 2024

Leo6Leo Jan 30, 2024 •

edited

Loading

creydr Jan 31, 2024

pierDipi Jan 31, 2024

Leo6Leo Jan 31, 2024

Trust-manager integration #7532

Trust-manager integration #7532

Conversation

pierDipi commented Jan 5, 2024 • edited Loading

Proposed Changes

Pre-review Checklist

knative-prow bot commented Jan 5, 2024

codecov bot commented Jan 8, 2024 • edited Loading

Codecov Report

pierDipi commented Jan 10, 2024

pierDipi commented Jan 10, 2024

creydr left a comment

Choose a reason for hiding this comment

pierDipi commented Jan 10, 2024

Leo6Leo Jan 30, 2024 • edited Loading

Choose a reason for hiding this comment

creydr Jan 31, 2024

Choose a reason for hiding this comment

pierDipi Jan 31, 2024

Choose a reason for hiding this comment

Leo6Leo Jan 31, 2024

Choose a reason for hiding this comment

pierDipi commented Jan 5, 2024 •

edited

Loading

codecov bot commented Jan 8, 2024 •

edited

Loading

Leo6Leo Jan 30, 2024 •

edited

Loading