TiDB Cluster/Client TLS support implementation in TiDB Operator

[weekface]

To support various certificate issuers and methods, for example: user-defined certificates, K8s builtin CA system or cert-manager, we will refactor the TiDB Cluster/Client TLS feature with new API and new usage.

First, we will change the Cluster/Client TLS API to:

tlsCluster:
  enabled: true

tidb:
  tlsClient:
    enabled: true

That is all the whole API we needed.

If the user set tlsCluster.enabled to true, then tidb-operator will assume that there are several Secrets named with: <cluster-name>-pd-cluster-secret, <cluster-name>-tikv-cluster-secret, <cluster-name>-tidb-cluster-secret, <cluser-name>-cluster-client-secret and other TiDB components's Secrets are created.

Each Secret data should have three keys: tls.crt, tls.key and ca.crt:

apiVersion: v1
kind: Secret
metadata:
  name: <secret-name>
data:
  tls.crt: <base64 decoded certificate data>
  tls.key: <base64 decode key data>
  ca.crt: <base64 decode ca data>

These Secrets can be created by the use manually, by K8s builtin CA system or by cert-manager. PD/TiKV/TiDB/... will use these Secrets to start server.

tidb-operator will not supply these certificates automatically.

There are several tasks:

• Remove old auto generated codes
  • remove TLS codes for client tls: Enable TLS For MySQL Clients #1867
  • remove TLS codes between components tls: TLS between TiDB components #1870
• TLS for MySQL Clients
  • TiDB Server/Client tls: Enable TLS For MySQL Clients #1867
  • TiDB add require-secure-transport option to TiDB Server *: support require-secure-transport startup option tidb#15341
  • TiDB Operator support require-secure-transport option
  • TiDB Initializer make tidb-initializer support TLS #1931
  • verify cert-based authentication manually
• TLS between TiDB components
  • PD/TiKV/TiDB tls: TLS between TiDB components #1870
  • Pump/Drainer
    • binlog: add tls in pump and drainer #1739
    • slowLogTailer: rotated slow logs not cleaned  #1779
    • Pump and Drainer always print http not https scheme, Log https as the scheme while TLS is enabled tidb-binlog#936
    • TiDB-Binlog doesn't support wildcard SAN? TiDB failed to write binlog while Pump TLS is enabled tidb-binlog#935
  • pd-ctl
  • BR backup: support br compatible with new TLS interface #1988
  • TiDB Monitor
    • PD/TiDB TidbMonitor failed to scrape PD/TiKV metrics while TLS is enabled #1917 fix prometheus scrape config issue while TLS is enabled #1919
    • TiKV support TLS status API Metrics page not using HTTPS after TLS enabled tikv/tikv#5340 server: support tls for the status server tikv/tikv#5393
    • TiDB Operator supports TiKV TLS status API tls for tikv metircs api #2137
• Documents
  • TLS for MySQL Clients
    • user-defined TLS
      • mysql-client
      • TidbInitializer
    • K8s built-in certificate issue system
      • mysql-client
      • TidbInitializer
    • cert-manager issue system
      • mysql-client
      • TidbInitializer
  • TLS for TiDB Components
    • user-defined TLS
      • PD/TiKV/TiDB/Monitor
      • Pump/Drainer
      • BR
    • K8s built-in certificate issue system
      • PD/TiKV/TiDB/Monitor
      • Pump/Drainer
      • BR
    • cert-manager issue system
      • PD/TiKV/TiDB/Monitor
      • Pump/Drainer
      • BR
• Test PD/TiDB/TiKV reload TLS certificate online
• Support allowed Common Name(CN)
  • *: support allowed Common Name(CN) tikv/pd#2241
  • config: let the cert-allowed-cn same with other components tikv/pd#2251
  • PD support only one CN: Failed to start PD when cert-allowed-cn is more than one #1998
  • PD CN cert issue: client certificate authentication issue when pd cert-allowed-cn is supplied #2002
  • TiDB Should started with advertize-addr issue: support starting tidb-server with -advertise-address parameter #1859
  • support common name check for tls tidb-binlog#934
  • server: support check the "CommanName" of tls-cert for status-port(http/grpc) tidb#15137
  • Security: support common names validation tikv/tikv#7134
  • TiDB Operator supports cert-allowed-cn cert-allowed-cn support #2061
  • can't connect to tidb status api: Can't connect to TiDB status API when tls and cluster-verify-cn is set #2047
  • failed to set pump cert-allowed-cn: failed to set cert-allowed-cn option for pump #2046
  • pd issue: issues when PD cert-allowed-cn was set #2048
• Add E2E test specs
• How to verify the Cluster/Client TLS is enabled correctly?
• Mydumper/Backup/BackupSchedule/Restore/DM(?) (after v1.1 GA)
• Lightning(Support TLS; Reduce the need of config.toml in integration tests tidb-lightning#270) and Importer(*: support TLS tikv/importer#40) (after v1.1 GA)
• tidb probe problem: TiDB Pod Probe doesn't work with TLS #2132 change tidb readiness probe to TCPSocket 4000 port #2139
• cert-manager problem: TLS can't work while using cert-manager v0.14.1 #2127
• tidb-operator should use renew client cert: tidb-operator should use the latest TLS client certificate automatically when renewing TLS #2138

Low priority or do not do issues:

• mariadb-client problem (please use oracle mysql-client 5.7 instead)
  • tidb server failed to validity client certificate period
  • wrong client cert can connect to tidb server
• tikv-ctl tikv-ctl failed to connect to TiKV Server when TLS is enabled tikv/tikv#7016
• tidb-ctl How to use tidb-ctl to connect a TiDB Cluster with TLS enabled tidb#15173

[github-actions]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 15 days