[DOP-15764] - add jdbc option classes

MobileTeleSystems · May 21, 2024 · 86a56f8 · 86a56f8
1 parent aa2b753
commit 86a56f8
Show file tree

Hide file tree

Showing 37 changed files with 432 additions and 120 deletions.
diff --git a/docs/changelog/next_release/277.feature.rst b/docs/changelog/next_release/277.feature.rst
@@ -0,0 +1 @@
+Add ``ReadOptions``, ``WriteOptions``, ``SQLOptions``, ``FetchOptions``, ``ExecuteOptions`` option classes for each JDBC connector.
diff --git a/docs/conf.py b/docs/conf.py
@@ -57,6 +57,7 @@
     "sphinxcontrib.towncrier",  # provides `towncrier-draft-entries` directive
     "sphinxcontrib.plantuml",
     "sphinx.ext.extlinks",
+    "sphinxcontrib.autodoc_pydantic",
     "sphinx_favicon",
 ]
 numpydoc_show_class_members = False

diff --git a/docs/connection/db_connection/clickhouse/execute.rst b/docs/connection/db_connection/clickhouse/execute.rst
@@ -20,7 +20,7 @@ Use ``Clickhouse.fetch``
 Use this method to perform some ``SELECT`` query which returns **small number or rows**, like reading
 Clickhouse config, or reading data from some reference table. Method returns Spark DataFrame.
 
-Method accepts :obj:`FetchOptions <onetl.connection.db_connection.jdbc_mixin.options.JDBCFetchOptions>`.
+Method accepts :obj:`ClickhouseFetchOptions <onetl.connection.db_connection.clickhouse.options.ClickhouseFetchOptions>`.
 
 Connection opened using this method should be then closed with ``connection.close()`` or ``with connection:``.
 
@@ -60,7 +60,7 @@ Use ``Clickhouse.execute``
 
 Use this method to execute DDL and DML operations. Each method call runs operation in a separated transaction, and then commits it.
 
-Method accepts :obj:`ExecuteOptions <onetl.connection.db_connection.jdbc_mixin.options.JDBCExecuteOptions>`.
+Method accepts :obj:`ClickhouseExecuteOptions <onetl.connection.db_connection.clickhouse.options.ClickhouseExecuteOptions>`.
 
 Connection opened using this method should be then closed with ``connection.close()`` or ``with connection:``.
 
@@ -111,15 +111,13 @@ So it should **NOT** be used to read large amounts of data. Use :ref:`DBReader <
 Options
 -------
 
-.. currentmodule:: onetl.connection.db_connection.jdbc_mixin.options
+.. currentmodule:: onetl.connection.db_connection.clickhouse.options
 
-.. autopydantic_model:: JDBCFetchOptions
+.. autopydantic_model:: ClickhouseFetchOptions
+    :inherited-members: JDBCFetchOptions
     :member-order: bysource
-    :model-show-field-summary: false
-    :field-show-constraints: false
 
 
-.. autopydantic_model:: JDBCExecuteOptions
+.. autopydantic_model:: ClickhouseExecuteOptions
+    :inherited-members: JDBCExecuteOptions
     :member-order: bysource
-    :model-show-field-summary: false
-    :field-show-constraints: false
diff --git a/docs/connection/db_connection/clickhouse/read.rst b/docs/connection/db_connection/clickhouse/read.rst
@@ -22,7 +22,7 @@ Supported DBReader features
 * * ✅︎ :ref:`incremental-batch-strategy`
 * ❌ ``hint`` (is not supported by Clickhouse)
 * ❌ ``df_schema``
-* ✅︎ ``options`` (see :obj:`JDBCReadOptions <onetl.connection.db_connection.jdbc.options.JDBCReadOptions>`)
+* ✅︎ ``options`` (see :obj:`ClickhouseReadOptions <onetl.connection.db_connection.clickhouse.options.ClickhouseReadOptions>`)
 
 Examples
 --------
@@ -85,9 +85,8 @@ Especially if there are indexes or partitions for columns used in ``where`` clau
 Options
 -------
 
-.. currentmodule:: onetl.connection.db_connection.jdbc_connection.options
+.. currentmodule:: onetl.connection.db_connection.clickhouse.options
 
-.. autopydantic_model:: JDBCReadOptions
+.. autopydantic_model:: ClickhouseReadOptions
+    :inherited-members: JDBCReadOptions
     :member-order: bysource
-    :model-show-field-summary: false
-    :field-show-constraints: false
diff --git a/docs/connection/db_connection/clickhouse/sql.rst b/docs/connection/db_connection/clickhouse/sql.rst
@@ -70,6 +70,10 @@ Especially if there are indexes or partitions for columns used in ``where`` clau
 Options
 -------
 
-.. currentmodule:: onetl.connection.db_connection.jdbc_connection.options
+.. currentmodule:: onetl.connection.db_connection.clickhouse.options
 
-.. autopydantic_model:: JDBCSQLOptions
+.. autopydantic_model:: ClickhouseSQLOptions
+    :inherited-members: JDBCOptions
+    :member-order: bysource
+    :model-show-field-summary: false
+    :field-show-constraints: false
diff --git a/docs/connection/db_connection/greenplum/execute.rst b/docs/connection/db_connection/greenplum/execute.rst
@@ -20,7 +20,7 @@ Use ``Greenplum.fetch``
 Use this method to perform some ``SELECT`` query which returns **small number or rows**, like reading
 Greenplum config, or reading data from some reference table. Method returns Spark DataFrame.
 
-Method accepts :obj:`FetchOptions <onetl.connection.db_connection.jdbc_mixin.options.JDBCFetchOptions>`.
+Method accepts :obj:`GreenplumFetchOptions <onetl.connection.db_connection.greenplum.options.GreenplumFetchOptions>`.
 
 Connection opened using this method should be then closed with ``connection.close()`` or ``with connection:``.
 
@@ -60,7 +60,7 @@ Use ``Greenplum.execute``
 
 Use this method to execute DDL and DML operations. Each method call runs operation in a separated transaction, and then commits it.
 
-Method accepts :obj:`ExecuteOptions <onetl.connection.db_connection.jdbc_mixin.options.JDBCExecuteOptions>`.
+Method accepts :obj:`GreenplumExecuteOptions <onetl.connection.db_connection.greenplum.options.GreenplumExecuteOptions>`.
 
 Connection opened using this method should be then closed with ``connection.close()`` or ``with connection:``.
 
@@ -143,15 +143,13 @@ The only port used while interacting with Greenplum in this case is ``5432`` (Gr
 Options
 -------
 
-.. currentmodule:: onetl.connection.db_connection.jdbc_mixin.options
+.. currentmodule:: onetl.connection.db_connection.greenplum.options
 
-.. autopydantic_model:: JDBCFetchOptions
+.. autopydantic_model:: GreenplumFetchOptions
+    :inherited-members: JDBCFetchOptions
     :member-order: bysource
-    :model-show-field-summary: false
-    :field-show-constraints: false
 
 
-.. autopydantic_model:: JDBCExecuteOptions
+.. autopydantic_model:: GreenplumExecuteOptions
+    :inherited-members: JDBCExecuteOptions
     :member-order: bysource
-    :model-show-field-summary: false
-    :field-show-constraints: false
diff --git a/docs/connection/db_connection/mssql/execute.rst b/docs/connection/db_connection/mssql/execute.rst
@@ -20,7 +20,7 @@ Use ``MSSQL.fetch``
 Use this method to perform some ``SELECT`` query which returns **small number or rows**, like reading
 MSSQL config, or reading data from some reference table. Method returns Spark DataFrame.
 
-Method accepts :obj:`FetchOptions <onetl.connection.db_connection.jdbc_mixin.options.JDBCFetchOptions>`.
+Method accepts :obj:`MSSQLFetchOptions <onetl.connection.db_connection.mssql.options.MSSQLFetchOptions>`.
 
 Connection opened using this method should be then closed with ``connection.close()`` or ``with connection:``.
 
@@ -59,7 +59,7 @@ Use ``MSSQL.execute``
 
 Use this method to execute DDL and DML operations. Each method call runs operation in a separated transaction, and then commits it.
 
-Method accepts :obj:`ExecuteOptions <onetl.connection.db_connection.jdbc_mixin.options.JDBCExecuteOptions>`.
+Method accepts :obj:`MSSQLExecuteOptions <onetl.connection.db_connection.mssql.options.MSSQLExecuteOptions>`.
 
 Connection opened using this method should be then closed with ``connection.close()`` or ``with connection:``.
 
@@ -103,15 +103,12 @@ Examples
 Options
 -------
 
-.. currentmodule:: onetl.connection.db_connection.jdbc_mixin.options
+.. currentmodule:: onetl.connection.db_connection.mssql.options
 
-.. autopydantic_model:: JDBCFetchOptions
+.. autopydantic_model:: MSSQLFetchOptions
+    :inherited-members: JDBCFetchOptions
     :member-order: bysource
-    :model-show-field-summary: false
-    :field-show-constraints: false
 
-
-.. autopydantic_model:: JDBCExecuteOptions
+.. autopydantic_model:: MSSQLExecuteOptions
+    :inherited-members: JDBCExecuteOptions
     :member-order: bysource
-    :model-show-field-summary: false
-    :field-show-constraints: false
diff --git a/docs/connection/db_connection/mssql/read.rst b/docs/connection/db_connection/mssql/read.rst
@@ -22,7 +22,7 @@ Supported DBReader features
 * * ✅︎ :ref:`incremental-batch-strategy`
 * ❌ ``hint`` (MSSQL does support hints, but DBReader not, at least for now)
 * ❌ ``df_schema``
-* ✅︎ ``options`` (see :obj:`JDBCReadOptions <onetl.connection.db_connection.jdbc.options.JDBCReadOptions>`)
+* ✅︎ ``options`` (see :obj:`MSSQLReadOptions <onetl.connection.db_connection.mssql.options.MSSQLReadOptions>`)
 
 Examples
 --------
@@ -85,9 +85,8 @@ Especially if there are indexes or partitions for columns used in ``where`` clau
 Options
 -------
 
-.. currentmodule:: onetl.connection.db_connection.jdbc_connection.options
+.. currentmodule:: onetl.connection.db_connection.mssql.options
 
-.. autopydantic_model:: JDBCReadOptions
+.. autopydantic_model:: MSSQLReadOptions
+    :inherited-members: JDBCReadOptions
     :member-order: bysource
-    :model-show-field-summary: false
-    :field-show-constraints: false
diff --git a/docs/connection/db_connection/mssql/sql.rst b/docs/connection/db_connection/mssql/sql.rst
@@ -43,7 +43,7 @@ Examples
         WHERE
             key = 'something'
         """,
-        options=MSSQL.QLOptions(
+        options=MSSQL.SQLOptions(
             partition_column="id",
             num_partitions=10,
             lower_bound=0,
@@ -70,6 +70,10 @@ Especially if there are indexes or partitions for columns used in ``where`` clau
 Options
 -------
 
-.. currentmodule:: onetl.connection.db_connection.jdbc_connection.options
+.. currentmodule:: onetl.connection.db_connection.mssql.options
 
-.. autopydantic_model:: JDBCSQLOptions
+.. autopydantic_model:: MSSQLSQLOptions
+    :inherited-members: JDBCOptions
+    :member-order: bysource
+    :model-show-field-summary: false
+    :field-show-constraints: false
diff --git a/docs/connection/db_connection/mysql/execute.rst b/docs/connection/db_connection/mysql/execute.rst
@@ -20,7 +20,7 @@ Use ``MySQL.fetch``
 Use this method to perform some ``SELECT`` query which returns **small number or rows**, like reading
 MySQL config, or reading data from some reference table. Method returns Spark DataFrame.
 
-Method accepts :obj:`FetchOptions <onetl.connection.db_connection.jdbc_mixin.options.JDBCFetchOptions>`.
+Method accepts :obj:`MySQLFetchOptions <onetl.connection.db_connection.mysql.options.MySQLFetchOptions>`.
 
 Connection opened using this method should be then closed with ``connection.close()`` or ``with connection:``.
 
@@ -60,7 +60,7 @@ Use ``MySQL.execute``
 
 Use this method to execute DDL and DML operations. Each method call runs operation in a separated transaction, and then commits it.
 
-Method accepts :obj:`ExecuteOptions <onetl.connection.db_connection.jdbc_mixin.options.JDBCExecuteOptions>`.
+Method accepts :obj:`MySQLExecuteOptions <onetl.connection.db_connection.mysql.options.MySQLExecuteOptions>`.
 
 Connection opened using this method should be then closed with ``connection.close()`` or ``with connection:``.
 
@@ -104,15 +104,13 @@ Examples
 Options
 -------
 
-.. currentmodule:: onetl.connection.db_connection.jdbc_mixin.options
+.. currentmodule:: onetl.connection.db_connection.mysql.options
 
-.. autopydantic_model:: JDBCFetchOptions
+.. autopydantic_model:: MySQLFetchOptions
+    :inherited-members: JDBCFetchOptions
     :member-order: bysource
-    :model-show-field-summary: false
-    :field-show-constraints: false
 
 
-.. autopydantic_model:: JDBCExecuteOptions
+.. autopydantic_model:: MySQLExecuteOptions
+    :inherited-members: JDBCExecuteOptions
     :member-order: bysource
-    :model-show-field-summary: false
-    :field-show-constraints: false
diff --git a/docs/connection/db_connection/mysql/read.rst b/docs/connection/db_connection/mysql/read.rst
@@ -22,7 +22,7 @@ Supported DBReader features
 * * ✅︎ :ref:`incremental-batch-strategy`
 * ✅︎ ``hint`` (see `official documentation <https://dev.mysql.com/doc/refman/en/optimizer-hints.html>`_)
 * ❌ ``df_schema``
-* ✅︎ ``options`` (see :obj:`JDBCReadOptions <onetl.connection.db_connection.jdbc.options.JDBCReadOptions>`)
+* ✅︎ ``options`` (see :obj:`MySQLReadOptions <onetl.connection.db_connection.mysql.options.MySQLReadOptions>`)
 
 Examples
 --------
@@ -87,9 +87,8 @@ Especially if there are indexes for columns used in ``where`` clause.
 Options
 -------
 
-.. currentmodule:: onetl.connection.db_connection.jdbc_connection.options
+.. currentmodule:: onetl.connection.db_connection.mysql.options
 
-.. autopydantic_model:: JDBCReadOptions
+.. autopydantic_model:: MySQLReadOptions
+    :inherited-members: JDBCReadOptions
     :member-order: bysource
-    :model-show-field-summary: false
-    :field-show-constraints: false
diff --git a/docs/connection/db_connection/mysql/sql.rst b/docs/connection/db_connection/mysql/sql.rst
@@ -71,6 +71,10 @@ Especially if there are indexes or partitions for columns used in ``where`` clau
 Options
 -------
 
-.. currentmodule:: onetl.connection.db_connection.jdbc_connection.options
+.. currentmodule:: onetl.connection.db_connection.mysql.options
 
-.. autopydantic_model:: JDBCSQLOptions
+.. autopydantic_model:: MySQLSQLOptions
+    :inherited-members: JDBCOptions
+    :member-order: bysource
+    :model-show-field-summary: false
+    :field-show-constraints: false
diff --git a/docs/connection/db_connection/oracle/execute.rst b/docs/connection/db_connection/oracle/execute.rst
@@ -20,7 +20,7 @@ Use ``Oracle.fetch``
 Use this method to execute some ``SELECT`` query which returns **small number or rows**, like reading
 Oracle config, or reading data from some reference table. Method returns Spark DataFrame.
 
-Method accepts :obj:`FetchOptions <onetl.connection.db_connection.jdbc_mixin.options.JDBCFetchOptions>`.
+Method accepts :obj:`OracleFetchOptions <onetl.connection.db_connection.oracle.options.OracleFetchOptions>`.
 
 Connection opened using this method should be then closed with ``connection.close()`` or ``with connection:``.
 
@@ -60,7 +60,7 @@ Use ``Oracle.execute``
 
 Use this method to execute DDL and DML operations. Each method call runs operation in a separated transaction, and then commits it.
 
-Method accepts :obj:`ExecuteOptions <onetl.connection.db_connection.jdbc_mixin.options.JDBCExecuteOptions>`.
+Method accepts :obj:`OracleExecuteOptions <onetl.connection.db_connection.oracle.options.OracleExecuteOptions>`.
 
 Connection opened using this method should be then closed with ``connection.close()`` or ``with connection:``.
 
@@ -104,15 +104,13 @@ Examples
 Options
 -------
 
-.. currentmodule:: onetl.connection.db_connection.jdbc_mixin.options
+.. currentmodule:: onetl.connection.db_connection.oracle.options
 
-.. autopydantic_model:: JDBCFetchOptions
+.. autopydantic_model:: OracleFetchOptions
+    :inherited-members: JDBCFetchOptions
     :member-order: bysource
-    :model-show-field-summary: false
-    :field-show-constraints: false
 
 
-.. autopydantic_model:: JDBCExecuteOptions
+.. autopydantic_model:: OracleExecuteOptions
+    :inherited-members: JDBCExecuteOptions
     :member-order: bysource
-    :model-show-field-summary: false
-    :field-show-constraints: false
diff --git a/docs/connection/db_connection/oracle/read.rst b/docs/connection/db_connection/oracle/read.rst
@@ -22,7 +22,7 @@ Supported DBReader features
 * * ✅︎ :ref:`incremental-batch-strategy`
 * ✅︎ ``hint`` (see `official documentation <https://docs.oracle.com/cd/B10500_01/server.920/a96533/hintsref.htm>`_)
 * ❌ ``df_schema``
-* ✅︎ ``options`` (see :obj:`JDBCReadOptions <onetl.connection.db_connection.jdbc.options.JDBCReadOptions>`)
+* ✅︎ ``options`` (see :obj:`OracleReadOptions <onetl.connection.db_connection.oracle.options.OracleReadOptions>`)
 
 Examples
 --------
@@ -87,9 +87,8 @@ Especially if there are indexes or partitions for columns used in ``where`` clau
 Options
 -------
 
-.. currentmodule:: onetl.connection.db_connection.jdbc_connection.options
+.. currentmodule:: onetl.connection.db_connection.oracle.options
 
-.. autopydantic_model:: JDBCReadOptions
+.. autopydantic_model:: OracleReadOptions
+    :inherited-members: JDBCReadOptions
     :member-order: bysource
-    :model-show-field-summary: false
-    :field-show-constraints: false
diff --git a/docs/connection/db_connection/oracle/sql.rst b/docs/connection/db_connection/oracle/sql.rst
@@ -71,6 +71,10 @@ Especially if there are indexes or partitions for columns used in ``where`` clau
 Options
 -------
 
-.. currentmodule:: onetl.connection.db_connection.jdbc_connection.options
+.. currentmodule:: onetl.connection.db_connection.oracle.options
 
-.. autopydantic_model:: JDBCSQLOptions
+.. autopydantic_model:: OracleSQLOptions
+    :inherited-members: JDBCOptions
+    :member-order: bysource
+    :model-show-field-summary: false
+    :field-show-constraints: false
diff --git a/docs/connection/db_connection/postgres/execute.rst b/docs/connection/db_connection/postgres/execute.rst
@@ -20,7 +20,7 @@ Use ``Postgres.fetch``
 Use this method to execute some ``SELECT`` query which returns **small number or rows**, like reading
 Postgres config, or reading data from some reference table. Method returns Spark DataFrame.
 
-Method accepts :obj:`FetchOptions <onetl.connection.db_connection.jdbc_mixin.options.JDBCFetchOptions>`.
+Method accepts :obj:`PostgresFetchOptions <onetl.connection.db_connection.postgres.options.PostgresFetchOptions>`.
 
 Connection opened using this method should be then closed with ``connection.close()`` or ``with connection:``.
 
@@ -58,7 +58,7 @@ Use ``Postgres.execute``
 
 Use this method to execute DDL and DML operations. Each method call runs operation in a separated transaction, and then commits it.
 
-Method accepts :obj:`ExecuteOptions <onetl.connection.db_connection.jdbc_mixin.options.JDBCExecuteOptions>`.
+Method accepts :obj:`PostgresExecuteOptions <onetl.connection.db_connection.postgres.options.PostgresExecuteOptions>`.
 
 Connection opened using this method should be then closed with ``connection.close()`` or ``with connection:``.
 
@@ -102,15 +102,13 @@ Examples
 Options
 -------
 
-.. currentmodule:: onetl.connection.db_connection.jdbc_mixin.options
+.. currentmodule:: onetl.connection.db_connection.postgres.options
 
-.. autopydantic_model:: JDBCFetchOptions
+.. autopydantic_model:: PostgresFetchOptions
+    :inherited-members: JDBCFetchOptions
     :member-order: bysource
-    :model-show-field-summary: false
-    :field-show-constraints: false
 
 
-.. autopydantic_model:: JDBCExecuteOptions
+.. autopydantic_model:: PostgresExecuteOptions
+    :inherited-members: JDBCExecuteOptions
     :member-order: bysource
-    :model-show-field-summary: false
-    :field-show-constraints: false