authentication-mechanisms-gcp.md

Configure Authentication Mechanism for GCP

Authentication Mechanism for the Control Plane

This is the manual auth configuration for the Control Plane. Refer to the installation guide for automated scripts.

For both methods (Workload Identity and Kubernetes Secret), the first manual step is creating a Google Cloud Service Account with the appropriate permissions needed for the control plane to manage GCP resources.

You need to create a new Google Cloud Service Account named events-controller-gsa with the following command:

gcloud iam service-accounts create events-controller-gsa

Then, give that Google Cloud Service Account permissions on your project. The actual permissions needed will depend on the resources you are planning to use. The Table below enumerates such permissions:

Resource / Functionality	Roles
CloudPubSubSource	roles/pubsub.editor
CloudStorageSource	roles/storage.admin
CloudSchedulerSource	roles/cloudscheduler.admin
CloudAuditLogsSource	roles/pubsub.admin, roles/logging.configWriter, roles/logging.privateLogViewer
CloudBuildSource	roles/pubsub.subscriber
Channel	roles/pubsub.editor
PullSubscription	roles/pubsub.editor
Topic	roles/pubsub.editor

In this guide, and for the sake of simplicity, we will just grant roles/owner privileges to the Google Cloud Service Account, which encompasses all of the above plus some other permissions. Note that if you prefer finer-grained privileges, you can just grant the ones described in the Table. Also, you can refer to managing multiple projects in case you want your Google Cloud Service Account to manage multiple projects.

gcloud projects add-iam-policy-binding $PROJECT_ID \
--member=serviceAccount:events-controller-gsa@$PROJECT_ID.iam.gserviceaccount.com \
--role roles/owner

Option 1 (Recommended): Workload Identity

1. Enable Workload Identity.

   1. Ensure that you have enabled the Cloud IAM Service Account Credentials API.

      gcloud services enable iamcredentials.googleapis.com

   2. If you didn't enable Workload Identity when you created your cluster, run the following commands:

      Modify the cluster to enable Workload Identity first:

      gcloud container clusters update $CLUSTER_NAME \
        --workload-pool=$PROJECT_ID.svc.id.goog

      Enable GKE_METADATA for your node pool(s).

       pools=$(gcloud container node-pools list --cluster=${CLUSTER_NAME} --format="value(name)")
       while read -r pool_name
       do
         gcloud container node-pools update "${pool_name}" \
           --cluster=${CLUSTER_NAME} \
           --workload-metadata=GKE_METADATA
       done <<<"${pools}"

      NOTE: These commands may take a long time to finish. Check Enable Workload Identity on an existing cluster for more information.

2. Create a Kubernetes Service Account ksa-name in the namespace your resources will reside. If you are configuring Authentication Mechanism for the Control Plane, you can skip this step, and directly use Kubernetes Service Account controller which is already in the Control Plane namespace cloud-run-events

   kubectl create serviceaccount ksa-name -n ksa-namespace

3. Bind the Kubernetes Service Account ksa-name with Google Cloud Service Account.

   MEMBER=serviceAccount:$PROJECT_ID.svc.id.goog[ksa-namespace/ksa-name]
   
   gcloud iam service-accounts add-iam-policy-binding \
   --role roles/iam.workloadIdentityUser \
   --member $MEMBER gsa-name@$PROJECT_ID.iam.gserviceaccount.com

   If you are configuring Authentication Mechanism for the Control Plane, you can replace ksa-namespace with cloud-run-events, ksa-name with controller, and gsa-name with events-controller-gsa

4. Annotate the Kubernetes Service Account ksa-name.

   kubectl annotate serviceaccount ksa-name iam.gke.io/gcp-service-account=gsa-name@$PROJECT_ID.iam.gserviceaccount.com \
   --namespace ksa-namespace

   If you are configuring Authentication Mechanism for the Control Plane, you can replace ksa-namespace with cloud-run-events, ksa-name with controller, and gsa-name with events-controller-gsa

Option 2: Kubernetes Secrets

1. Download a new JSON private key for that Service Account. Be sure not to check this key into source control!

   gcloud iam service-accounts keys create cloud-run-events.json \
     --iam-account=events-controller-gsa@$PROJECT_ID.iam.gserviceaccount.com

2. Create a Secret on the Kubernetes cluster in the cloud-run-events namespace with the downloaded key:

   kubectl --namespace cloud-run-events create secret generic google-cloud-key --from-file=key.json=cloud-run-events.json

   Note that google-cloud-key and key.json are default values expected by our control plane.

Authentication Mechanism for the Sources Data Plane

This is the manual auth configuration for the sources data plane using the Google Cloud Service Account events-sources-gsa as the credential. Refer to Installing a Service Account for the Data Plane for automated scripts. We recommend using a distinct service account for the broker data plane named events-broker-gsa. Follow Installing GCP Broker to set up the broker.

Option 1: Use Workload Identity

1. We assume that you have already enabled Workload Identity in your cluster.
2. There are two scenarios to leverage Workload Identity for resources in the Data Plane:

• Non-default scenario:

  Using the Google Cloud Service Account events-sources-gsa you created in Installing a Service Account for the Data Plane and using Option 1 (Recommended): Workload Identity in Authentication Mechanism for GCP to configure Workload Identity in the namespace your resources will reside. (You may notice that this link is pointing to the manual Workload Identity configuration in the Control Plane. Non-default scenario Workload Identity configuration in the Data Plane is similar to the manual Workload Identity configuration in the Control Plane)

  You will have a Kubernetes Service Account after the above configuration, which is bound to the Google Cloud Service Account events-sources-gsa. Remember to put this Kubernetes Service Account name as the spec.serviceAccountName when you create resources in the example.

• Default scenario:

  Instead of manually configuring Workload Identity with Authentication Mechanism for GCP, you can authorize the Controller to configure Workload Identity for you.

  You need to grant iam.serviceAccountAdmin permission of the Google Cloud Service Account events-sources-gsa you created in Installing a Service Account for the Data Plane to the Control Plane's Google Cloud Service Account events-controller-gsa by:

  gcloud iam service-accounts add-iam-policy-binding \
   events-sources-gsa@$PROJECT_ID.iam.gserviceaccount.com  \
   --member=serviceAccount:events-controller-gsa@$PROJECT_ID.iam.gserviceaccount.com \
   --role roles/iam.serviceAccountAdmin

  Then, modify clusterDefaults in ConfigMap config-gcp-auth.

  You can directly edit the ConfigMap by:

  kubectl edit configmap config-gcp-auth -n cloud-run-events

  and add workloadIdentityMapping in clusterDefaults:

  default-auth-config: |
    clusterDefaults:
      workloadIdentityMapping:
        serviceAccountName: sources
        sources: events-sources-gsa@$PROJECT_ID.iam.gserviceaccount.com

  When updating the configuration, note that default-auth-config is nested under data. If you encounter an error, you are likely attempting to modify the example configuration in _example.

  Here, sources refers to a Kubernetes Service Account bound to the Google Cloud Service Account events-sources-gsa. Remember to put this Kubernetes Service Account name as the spec.serviceAccountName when you create resources in the example.

  Kubernetes Service Account sources doesn't need to exist in a specific namespace. Once it is set in the ConfigMap config-gcp-auth, the Control Plane will create it for you and configure the corresponding Workload Identity relationship between the Kubernetes Service Account sources and the Google Cloud Service Account events-sources-gsa when you create resources using the Kubernetes Service Account sources.

  A Condition WorkloadIdentityConfigured will show up under resources' Status, indicating the Workload Identity configuration status.

  Note: The Controller currently doesn’t perform any access control checks, as a result, any user who can create a resource can get access to the Google Cloud Service Account which grants the iam.serviceAccountAdmin permission to the Controller. As an example, if you followed the instructions above, then any user that can make a Knative-GCP source or Channel (e.g. CloudAuditLogsSource, CloudPubSubSource, etc.) can cause the Kubernetes Service Account sources to be created. If they can also create Pods in that namespace, then they can make a Pod that uses the Google Service Account events-sources-gsa credentials.

Option 2. Export Service Account Keys And Store Them as Kubernetes Secrets

1. Download a new JSON private key for the Google Cloud Service Account events-sources-gsa created in Installing a Service Account for the Data Plane. Be sure not to check this key into source control!

   gcloud iam service-accounts keys create events-sources-key.json \
     --iam-account=events-sources-gsa@$PROJECT_ID.iam.gserviceaccount.com

2. Create a secret on the Kubernetes cluster with the downloaded key. Remember to create the secret in the namespace your resources will reside. The example below does so in the default namespace.

   kubectl --namespace default create secret generic google-cloud-key --from-file=key.json=events-sources-key.json

   google-cloud-key and key.json are default values expected by our resources.

3. Cleaning Up:

   1. Delete the secret

      kubectl --namespace default delete secret google-cloud-key