Support TLS connections when only ssl-ca is set

[kolbe]

Feature Request

Is your feature request related to a problem? Please describe:
In some cases, an upstream (MySQL, etc.) server may be configured to require SSL/TLS, but not in an environment where client certificates are available.

This can be useful in situations where MySQL is configured using a specific CA key, and only clients that trust that same CA key are allowed to connect.

Describe the feature you'd like:
If only ssl-ca is set in the source configuration file, DM should try to connect using TLS without a client certificate.

[kolbe]

This functionality causes serious problems for environments like Azure Database for MySQL, which allows you to require TLS but does not provide client certificates.

[kolbe]

$ cat ~/source.yaml
source-id: "testdb"
from:
  host: "127.0.0.1"
  port: 3306
  user: "root"
  password: ""
  security:
    ssl-ca: "/Users/kolbe/Devel/mysql/mysql-5.7.17-macos10.12-x86_64/myCA.pem"

$ ./bin/dmctl --master-addr 127.0.0.1:8261 operate-source create ~/source.yaml
{
    "result": false,
    "msg": "[code=10001:class=database:scope=not-set:level=high], Message: database driver error, RawCause: Error 3159: Connections using insecure transport are prohibited while --require_secure_transport=ON., Workaround: Please check the database connection and the database config in configuration file.",
    "sources": [
    ]
}

$ cat ~/source.yaml
source-id: "testdb"
from:
  host: "127.0.0.1"
  port: 3306
  user: "root"
  password: ""
  security:
    ssl-ca: "/Users/kolbe/Devel/mysql/mysql-5.7.17-macos10.12-x86_64/myCA.pem"
    ssl-cert: "/Users/kolbe/Devel/mysql/mysql-5.7.17-macos10.12-x86_64/server.crt"
    ssl-key: "/Users/kolbe/Devel/mysql/mysql-5.7.17-macos10.12-x86_64/server.key"

$ ./bin/dmctl --master-addr 127.0.0.1:8261 operate-source create ~/source.yaml
{
    "result": true,
    "msg": "",
    "sources": [
        {
            "result": true,
            "msg": "",
            "source": "testdb",
            "worker": "worker1"
        }
    ]
}

[coderplay]

The error messages you pasted is not exactly the same as what I saw. Mine is like:

# /dmctl --master-addr ultra-dm-master-0:8261 operate-source create source.yaml
{
    "result": false,
    "msg": "[code=10001:class=database:scope=not-set:level=high], Message: database driver error, RawCause: Error 9002: SSL connection is required. Please specify SSL options and retry.\u0000, Workaround: Please check the database connection and the database config in configuration file.",
    "sources": [
    ]
}

[kolbe]

That’s right, it’s because you were connecting to Azure Database for MySQL. It emits a different error message than MySQL Server 5.7.

[coderplay]

the error was raised here: https://github.com/pingcap/dm/blob/v2.0.0/pkg/conn/basedb.go#L122

[coderplay]

Apparently we should patch

https://github.com/pingcap/dm/blob/v2.0.0/pkg/conn/basedb.go#L66-L83

[GMHDBJD]

Ok, we will check it.

[coderplay]

@GMHDBJD just left one comment on the PR.

[GMHDBJD]

Got it. We will fix it as soon as possible.

[coderplay]

The TLS issue isn't completely fixed. I got another similar error from worker side when I tried to create a migration task. Please update the TLS code for dumper/loader/syncer as well.

errror:

/ # /dmctl --master-addr ultra-dm-master-0:8261 start-task task.yaml
{
    "result": true,
    "msg": "",
    "sources": [
        {
            "result": false,
            "msg": "[code=38032:class=dm-master:scope=internal:level=high], Message: some error occurs in dm-worker: ErrCode:10001 ErrClass:\"database\" ErrScope:\"upstream\" ErrLevel:\"high\" Message:\"fail to initial unit Sync of subtask sql-history-migration : database driver error\" RawCause:\"Error 9002: SSL connection is required. Please specify SSL options and retry.\\000\" Workaround:\"Please check the database connection and the database config in configuration file.\" , Workaround: Please execute `query-status` to check status.",
            "source": "testdb",
            "worker": "ultra-dm-worker-0"
        }
    ]
}

[coderplay]

@kolbe I found from the source code that TiCDC has the same problem. Could you please file tickets for all the other TiDB projects that you can think of would have the same problem?

[kolbe]

@GMHDBJD can you please look at #1555 (comment) and be sure that all parts of DM code that connect to upstream data sources match this behavior of using SSL if only ssl-ca is set?

[lance6716]

closed by #1575