Skip to content

Latest commit

 

History

History

connect-jdbc

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
 
 
 
 
 
 

Kafka Connect JDBC test environment

This Docker Compose can be used to spin up an environment in which to explore and test the Kafka Connect JDBC source connector.

Table 1. Current environment

Confluent Platform

5.4.0

MySQL

8.0.13

Postgres

11.1

MS SQL Server

2017 (RTM-CU13) (KB4466404) - 14.0.3048.4 (X64)

Oracle

12.2.0.1.0

Getting a CLI session

  • KSQL

    docker exec --tty --interactive ksql-cli ksql http://ksql-server:8088
  • MySQL

    docker exec --tty --interactive mysql bash -c 'mysql -u root -p$MYSQL_ROOT_PASSWORD'
  • Postgres

    docker exec --tty --interactive postgres bash -c 'psql -U $POSTGRES_USER $POSTGRES_DB'
  • Oracle

    docker exec --tty --interactive oracle bash -c 'sqlplus sys/$ORACLE_PWD@localhost:1521/ORCLCDB as sysdba'
  • MS SQL Server

    docker exec --tty --interactive mssql bash -c '/opt/mssql-tools/bin/sqlcmd -l 30 -S localhost -U sa -P $SA_PASSWORD'

Example Connector Configuration

  • MySQL

    curl -X POST http://localhost:8083/connectors -H "Content-Type: application/json" -d '{
            "name": "jdbc_source_mysql_01",
            "config": {
                    "connector.class": "io.confluent.connect.jdbc.JdbcSourceConnector",
                    "connection.url": "jdbc:mysql://mysql:3306/demo",
                    "connection.user": "connect_user",
                    "connection.password": "asgard",
                    "topic.prefix": "mysql-01-",
                    "mode":"bulk",
                    "poll.interval.ms" : 3600000
                    }
            }'
  • Postgres

    curl -X POST http://localhost:8083/connectors -H "Content-Type: application/json" -d '{
            "name": "jdbc_source_postgres_01",
            "config": {
                    "connector.class": "io.confluent.connect.jdbc.JdbcSourceConnector",
                      "connection.url": "jdbc:postgresql://postgres:5432/postgres",
                    "connection.user": "connect_user",
                    "connection.password": "asgard",
                    "topic.prefix": "postgres-01-",
                    "mode":"bulk",
                    "poll.interval.ms" : 3600000
                    }
            }'
  • Oracle

    curl -X POST http://localhost:8083/connectors -H "Content-Type: application/json" -d '{
                    "name": "jdbc_source_oracle_01",
                    "config": {
                            "connector.class": "io.confluent.connect.jdbc.JdbcSourceConnector",
                            "connection.url": "jdbc:oracle:thin:@oracle:1521/ORCLPDB1",
                            "connection.user": "connect_user",
                            "connection.password": "asgard",
                            "topic.prefix": "oracle-01-",
                            "table.whitelist" : "NUM_TEST",
                            "mode":"bulk",
                            "poll.interval.ms" : 3600000
                            }
                    }'
  • MS SQL Server

    curl -X POST http://localhost:8083/connectors -H "Content-Type: application/json" -d '{
                    "name": "jdbc_source_mssql_01",
                    "config": {
                            "connector.class": "io.confluent.connect.jdbc.JdbcSourceConnector",
                            "connection.url": "jdbc:sqlserver://mssql:1433;databaseName=demo",
                            "connection.user": "connect_user",
                            "connection.password": "Asgard123",
                            "topic.prefix": "mssql-01-",
                            "table.whitelist" : "demo..num_test",
                            "mode":"bulk",
                            "poll.interval.ms" : 3600000
                            }
                    }'