Allow expoisng hive tables from metastore #189

yruslan · 2023-04-14T07:48:52Z

Closes #90, #187 , #188

This PR includes the following:

Allow exposing of metastore tables vis Hive as simply as adding hive table name to the metastore table definition.
'user' and 'password' options are made optional for JDBC connections since they are not needed for paswordless auth
Add the ability to specify connection timeouts to JDBC connections
Add (restore from SyncWatcher) table locking mechanism to ensure only one task instance is running across clusters.

…Native

…job runner.

…nnections.

…the same time.

jirifilip

Looks great!

yruslan added 8 commits April 11, 2023 16:07

#90 Add configuration for Hive table exposure for metastore tables.

6a83c59

#90 Add configuration option for imnoring failures.

963e10d

#90 Split Hive default multiformat and the actual config.

08808fa

Add the ability to specify JDBC connection timeout for Hive and JDBC …

23e7ef9

…Native

#90 Add the ability to expose metastore tables via Hive.

d69ce96

#90 Implement the exposure of metastore tables as Hive tables in the …

4a1396e

…job runner.

#187 Make JDBC username and password optional when setting up JDBC co…

050fa0a

…nnections.

#188 Use locking to ensure that only one task instance is running at …

6e85ad5

…the same time.

yruslan marked this pull request as ready for review April 14, 2023 07:55

yruslan requested a review from jirifilip as a code owner April 14, 2023 07:55

Fix the data type of Delta partition data type.

7977628

jirifilip approved these changes Apr 17, 2023

View reviewed changes

yruslan merged commit f6075a4 into main Apr 17, 2023

yruslan deleted the feature/90-allow-expoisng-hive-tables-from-metastore branch April 17, 2023 12:25

Provide feedback