Skip to content

Latest commit

 

History

History
287 lines (236 loc) · 13.6 KB

populating-configuration.md

File metadata and controls

287 lines (236 loc) · 13.6 KB
Raw

NOTE some step in this document might refer to Red Hat internal components that you do not have access to

Populating Configuration

The document describes how to prepare Fleet Manager to be able to start by populating its configurations.

For convenience, the

make setup/secrets/empty

target is provided. This target initializes empty files that are necessary for the service to boot. It can be used to quickly get the service running locally.

Follow the subsections to find information about how to configure these files properly.

Interacting with the Fleet Manager API

The Fleet Manager API requires OIDC Bearer tokens, which use the JWT format, for authentication. The token is passed as an Authorization HTTP header value.

To be able to perform requests to the Fleet Manager API, you first need to get a JWT token from the configured OIDC authentication server, which is Red Hat SSO (sso.redhat.com) by default. Assuming the default OIDC authentication server is being used, this can be performed by interacting with the OCM API. This can be easily done interacting with it through the ocm cli and retrieve OCM tokens using it. To do so:

  1. Login to your desired OCM environment via web console using your Red Hat account credentials. For example, for the OCM production environment, go to https://cloud.redhat.com and login.
  2. Get your OCM offline token by going to https://cloud.redhat.com/openshift/token
  3. Login to your desired OCM environment through the OCM CLI by providing the OCM offline token and environment information: 
    ocm login --token <ocm-offline-token> --url <ocm-api-url>
    <ocm-api-url> is the URL of the OCM API. Some shorthands can also be provided like production or staging
  4. Generate an OCM token by running: ocm token. The generated token is the token that should be used to perform a request to the Fleet Manager API. For example:
curl -H "Authorization: Bearer <result-of-ocm-token-command>" http://127.0.0.1:/8000/api/dinosaurs_mgmt

OCM tokens other than the OCM offline token have an expiration time so a new one will need to be generated when that happens

There are some additional steps needed if you want to be able to perform certain actions that have additional requirements. See the User Account & Organization Setup for more information

Setting up OCM tokens

The Fleet Manager itself requires the use of an OCM token so it can interact with OCM to perform management of Data Plane clusters.

In order for the Fleet Manager to do so, an OCM offline token should be configured. To do so, retrieve your OCM offline token and then configure it for Fleet Manager by running:

make ocm/setup OCM_OFFLINE_TOKEN=<your-retrieved-ocm-offline-token>

Allowing creation of Standard Dinosaur instances

Fleet Manager is able to create two types of Dinosaur instances:

  • Eval instances
    • Instances of this type are automatically deleted after 48 hours by default

      NOTE: This can be controlled by setting the --dinosaur-lifespan Fleet Manager binary CLI flag

    • All authenticated users that make use of the Fleet Manager can request the creation of a Dinosaur eval instance
    • There is a limit of one instance per user
  • Standard instances
    • Instances of this type are not automatically deleted
    • In order to be able to create an instance of this type, the user that is creating it must have enough quota

If you are not interested in making use of Standard Dinosaur instances you can skip this section. Otherwise, keep reading below.

As commented above, In order o be able to create an instance of this type, the user must have enough quota to do it. There are currently two ways to define quotas for users in Fleet Manager:

To select the type of quota to be used by Fleet Manager set the --quota-type which accepts either ams or quota-management-listFleet Manager binary CLI flag.

Setup AWS configuration

Fleet Manager interacts with AWS to provide the following functionalities:

  • To be able to create and manage Data Plane clusters in a specific AWS account by passing the needed credentials to OpenShift Cluster Management
  • To create AWS's Route53 DNS records in a specific AWS account. This records are DNS records that point to some routes related to Dinosaurs instances that are created.

    NOTE: The domain name used for this records can be configured by setting the domain name to be used for Dinosaur instances. This cane be done through the --dinosaur-domain-name Fleet Manager binary CLI flag For both functionalities, the same underlying AWS account is used.

In order for the Fleet Manager to be able to start, create the following files:

touch secrets/aws.accountid
touch secrets/aws.accesskey
touch secrets/aws.secretaccesskey
touch secrets/aws.route53accesskey
touch secrets/aws.route53secretaccesskey

If you need any of those functionalities keep reading. Otherwise, this section can be skipped.

To accomplish the previously mentioned functionalities Fleet Manager needs to be configured to interact with the AWS account. To do so, provide existing AWS IAM user credentials to the control plane by running:

AWS_ACCOUNT_ID=<aws-account-id> \
AWS_ACCESS_KEY=<aws-iam-user-access-key> \
AWS_SECRET_ACCESS_KEY=<aws-iam-user-secret-access-key> \
ROUTE53_ACCESS_KEY=<aws-iam-user-for-route-53-access-key> \
ROUTE53_SECRET_ACCESS_KEY=<aws-iam-user-for-route-53-secret-access-key> \
make aws/setup

NOTE: If you are in Red Hat, the following documentation might be useful to get the IAM user/s credentials

Setup additional SSO configuration

A different SSO server other than the default authentication server (Red Hat SSO) is needed in Fleet Manager to enable some functionalities:

  • To enable communication between the Fleetshard operator (in the data plane) and the Fleet Manager
  • To configure the OpenShift cluster identity provider in Data Plane clusters to give SRE access to them

This additional SSO server must be based on Keycloak.

In order for the Fleet Manager to be able to start, create the following files:

touch secrets/keycloak-service.clientId
touch secrets/keycloak-service.clientSecret
touch secrets/osd-idp-keycloak-service.clientId
touch secrets/osd-idp-keycloak-service.clientSecret

If you need any of those functionalities keep reading. Otherwise, this section can be skipped.

Two alternatives are offered here to use as the SSO Keycloak server:

  • Use MAS-SSO

    NOTE: MAS-SSO is soon going to be deprecated. Reach out to the MAS-Security team for info regarding this.

  • Run your own Keycloak server instance by following their getting-started guide. Please note, how to setup Keycloak is out of scope of this guide

To accomplish the previously mentioned functionalities Fleet Manager needs to be configured to interact with this additional SSO server.

Create/Have available two client-id/client-secret pair of credentials (called Keycloak Clients) in the SSO Keycloak server, one for each previously mentioned functionality, and then set Fleet Manager to use them by running the following command:

 SSO_CLIENT_ID="<sso-client-id>" \
 SSO_CLIENT_SECRET="<sso-client-secret>" \
 OSD_IDP_SSO_CLIENT_ID="<osd-idp-sso-client-id>" \
 OSD_IDP_SSO_CLIENT_SECRET="<osd-idp-sso-client-secret" \
 make keycloak/setup

Additionally, make sure you start the Fleet Manager server with the appropriate Keycloak SSO Realms and URL for this. See the Fleet Manager CLI feature flags for details on it.

Setup the data plane image pull secret

In the Data Plane cluster, the Dinosaur Operator and the FleetShard Deployments might reference container images that are located in authenticated container image registries.

Fleet Manager can be configured to send this authenticated container image registry information as a K8s Secret in kubernetes.io/.dockerconfigjson format.

In order for the Fleet Manager to be able to start, create the following file:

touch secrets/image-pull.dockerconfigjson

If you don't need to make use of this functionality you can skip this section. Otherwise, keep reading below.

To configure the Fleet Manager with this authenticated registry information so the previously mentioned Data Plane elements can pull container images from it:

  • Base-64 encode your Docker configuration file.
  • Copy the contents generated from the previous point into the secrets/image-pull.dockerconfigjson file

Setup the Observability stack secrets

Fleet Manager provides the ability to provide and configure monitoring metrics related to the Data Planes and its Dinosaur instances. To do so it makes use of what's called the (Observability Stack

To configure and make use of the Observability stack to offer monitoring metrics related to the Data Planes and its Dinosaur instances Fleet Manager needs to:

  • Access the Observability Stack's Observatorium service to retrieve and then offer the metrics as one of the Fleet Manager API endpoints
  • To send the credentials/information needed to setup the Observability Stack when it is installed in the Data Plane clusters

In order for the Fleet Manager to be able to start, create the following files:

	touch secrets/rhsso-logs.clientId
	touch secrets/rhsso-logs.clientSecret
	touch secrets/rhsso-metrics.clientId
	touch secrets/rhsso-metrics.clientSecret
	touch secrets/observability-config-access.token

If you are not interested in making use of this functionality you can skip this section. Otherwise, keep reading below.

The following command is used to setup the various secrets needed by the Observability stack:

OBSERVATORIUM_CONFIG_ACCESS_TOKEN="<observatorium-config-access-token> \
RHSSO_LOGS_CLIENT_ID=<rhsso-logs-client-id> \
RHSSO_LOGS_CLIENT_SECRET=<rhsso-logs-client-secret> \
RHSSO_METRICS_CLIENT_ID=<rhsso-metrics-client-id> \
RHSSO_METRICS_CLIENT_SECRET=<rhsso-metrics-client-secret> \
make observatorium/setup

The description of the previously shown parameters are:

  • OBSERVATORIUM_CONFIG_ACCESS_TOKEN: A GitHub token required to fetch the configuration from a private git repository. More details can be found here.
  • RHSSO_LOGS_CLIENT_ID, RHSSO_LOGS_CLIENT_SECRET: An OAuth's client-id/client-secret credentials pair to push logs to Observatorium. Used to authenticate via sso.redhat.com when logging via config.promtail.enabled is set in the observability configuration. Provided by the Observatorium team.
  • RHSSO_METRICS_CLIENT_ID, RHSSO_METRICS_CLIENT_SECRET: An OAuth's client-id/client-secret credentials pair to push metrics to Observatorium. Used to authenticate via sso.redhat.com when config.prometheus.observatorium is set in the observability configuration. Provided by the Observatorium team.

See Observability to learn more about Observatorium and the Observability Stack.

Setup a custom TLS certificate for Dinosaur Host URLs

When Fleet Manager creates Dinosaur instances, it can be configured to send a custom TLS certificate to associate to each one of the Dinosaur instances host URLs. That custom TLS certificate is sent to the data plane clusters where those instances are located.

In order for the Fleet Manager to be able to start, create the following files:

touch secrets/dinosaur-tls.crt
touch secrets/dinosaur-tls.key

If you need to setup a custom TLS certificate for the Dinosaur instances' host URLs keep reading. Otherwise, this section can be skipped.

To configure Fleet Manager so it sends the custom TLS certificate, provide the certificate and its corresponding key to the Fleet Manager by running the following command:

DINOSAUR_TLS_CERT=<dinosaur-tls-cert> \
DINOSAUR_TLS_KEY=<dinosaur-tls-key> \
make dinosaurcert/setup

NOTE: The certificate domain/s should match the URL endpoint domain if you want the certificate to be valid when accessing the endpoint NOTE: The expected Certificate and Key values are in PEM format, preserving the newlines

Additionally, make sure that the functionality is enabled by setting the --enable-dinosaur-external-certificate Fleet Manager binary CLI flag

Configure Sentry logging

Fleet Manager can be configured to send its logs to the Sentry logging service.

In order for the Fleet Manager to be able to start, create the following files:

touch secrets/sentry.key

If you want to use Sentry set the Sentry Secret key in the secrets/sentry.key previously created.

Additionally, make sure to set the Sentry URL endpoint and Sentry project when starting the Fleet Manager server. See Sentry-related CLI flags in Fleet Manager