Skip to content

acosta11/cf-k8s-networking

 
 

Repository files navigation

cf-k8s-networking

​ Routing and networking for Cloud Foundry running on Kubernetes.

Architecture Diagram of CF-K8s-Networking

Installation

Prerequisites

  • A Cloud Foundry deployment using Eirini for app workloads
  • kubectl installed and access to the Kubernetes cluster backing Eirini
  • kapp installed
  • ytt installed

Metacontroller

Istio

  • Install Istio to the Kubernetes cluster.

  • Include the istio-values.yaml in your Istio installation.

    Note: As an example, in our CI we are installing Istio via the deploy-istio.sh task.

  • Enable ​automatic sidecar injection by labeling the following namespaces with istio-injection=enabled: cf-system, cf-workloads, metacontroller:

    kubectl label namespaces cf-system cf-workloads metacontroller istio-injection=enabled --overwrite
  • Sidecars are required for automatic mTLS between workloads so it is important that this is enabled. Confirm that the namespaces are labeled correctly:

    $ kubectl get namespaces -l istio-injection=enabled
    
    NAME             STATUS   AGE
    cf-system        Active   50d
    cf-workloads     Active   50d
    metacontroller   Active   50d

CF-K8s-Networking

  1. cfroutesync needs to be able to authenticate with UAA and fetch routes from Cloud Controller. To do this you must override the following properties from install/ytt/networking/values.yaml. You can do this by creating a new file /tmp/values.yaml that contains the following information:

    #@data/values
    ---
    cfroutesync:
      ccCA: 'pem_encoded_cloud_controller_ca'
      ccBaseURL: 'https://api.example.com'
      uaaCA: 'pem_encoded_uaa_ca'
      uaaBaseURL: 'https://uaa.example.com'
      clientName: 'uaaClientName'
      clientSecret: 'uaaClientSecret'

    The UAA client specified by clientName is used for fetching routing data from Cloud Controller. It must have permission to access all routes and domains in the deployment. We recommend using a client with at least the cloud_controller.admin_read_only authority. For example, see the network-policy client in cf-deployment.

    As an example, for our dev environments we are using the generate_values.rb script to populate these values from the bbl-state.json and secrets in CredHub.

  2. Deploy the cf-k8s-networking CRDs and components using ytt and kapp:

    system_namespace="cf-system"
    
    ytt -f config/cfroutesync/ -f /tmp/values.yaml \
        -f cfroutesync/crds/routebulksync.yaml | \
        kapp deploy -n "${system_namespace}" -a cfroutesync \
        -f - \
        -y
  3. Update the Prometheus configuration so metrics from cf-k8s-networking can be queried.

    prometheus_file="$(mktemp -u).yml"
    kubectl get -n istio-system configmap prometheus -o yaml > ${prometheus_file}
    
    ytt \
     -f "config/cfroutesync/values.yaml" \
     -f "${prometheus_file}" \
     -f "config/deps/prometheus-config.yaml" | \
     kubectl apply -f -

    Note: you might need to restart Prometheus pod(s) in the istio-system namespace after updating the ConfigMap 🧐🥺

    kubectl -n istio-system delete pod -l app=prometheus

About

building a cloud foundry without gorouter....

Resources

License

Code of conduct

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Go 76.9%
  • Shell 19.0%
  • Dockerfile 2.8%
  • Other 1.3%