v1.6.0

What's Changed

• #221 Add support for transient metastore tables in #243

Full Changelog: v1.5.2...v1.6.0

v1.5.2

What's Changed

• #234 Allow custom email notification info in #236
• #237 Fix redundant schema tracking of metastore tables that have 'raw' format in #238
• #239 Add support for column comments in inilne transformations in #241

Full Changelog: v1.5.1...v1.5.2

v1.5.1

What's Changed

• #225 Log executor nodes in #226
• #227 Add support for verbose log suppression for Spark 3.3+. in #228
• #229 Allow passing transformer configuration to the constructor in #231
• #232 Add the ability to specify alternative path to Hive tables if it differs from the table path in #233

Full Changelog: v1.5.0...v1.5.1

v1.5.0

What's Changed

• #216 Split metastore unit tests into multiple files by @jirifilip in #216
• #166 Fix get_latest_available_date for delta format by @jirifilip in #217
• #204 Run Python transformations on Databricks by @jirifilip in #204
• #165 Read delta tables directly from catalog by @jirifilip in #218
• #222 Add file based source and metastore table format by @yruslan in #223

Full Changelog: v1.4.3...v1.5.0

v1.4.3

What's Changed

• Hotfix: make sure fields that have empty names are not part of Hive DDL. by @yruslan in #213

Full Changelog: v1.4.2...v1.4.3

v1.4.2

What's Changed

• #203 Add spark catalog support as one of Hive integration options by @yruslan in #206
• #208 Make AppRunner recreate AppContext on every run by @jirifilip in #208
• #209 Improve EnceladusSink to support Hive external table creation by @yruslan in #209
• #210 Improve StandardizationSink to support Delta as publish format by @yruslan in #210
• #205 Allows spreading object array elements among multiple config files. by @yruslan in #211

Full Changelog: v1.4.1...v1.4.2

v1.4.1

What's Changed

• #196 Make Hive enablement an explicit parameter. by @yruslan in #199
• #197 Add Spark application Id to email notifications email notifications by @yruslan in #200
• #198 Improve exceptions rendering in notifications by @yruslan in #200

Full Changelog: v1.4.0...v1.4.1

v1.4.0

What's Changed

• #192 Add ability for a data source to be validated and return validation errors and warnings by @yruslan in #193
• #189 Allow expoisng hive tables from metastore by @yruslan in #189
• #190 Fix failed tasks log level. by @yruslan in #191

Full Changelog: v1.3.0...v1.4.0

v1.3.0

What's Changed

• #178 Add the workaround for the file appending issue. by @yruslan in #179
• #175 New Source and Sink interfaces by @yruslan in #180
• #182 Add custom query type by @yruslan in #183
• #184 Add the method to the Source trait by @yruslan in #185

Full Changelog: v1.2.2...v1.3.0

v1.2.2

What's Changed

• #132 Make command line sink more generic by @yruslan in #142
• #146 Fix failing tests from extras module on Windows. by @jirifilip in #147
• #145 Add an option for failing the pipeline if no data is available at the source by @yruslan in #148
• #150 Updated Spark Application Name (Syncwatcher -> Pramen) by @dwfchu in #150
• #153 Fix pipeline does not publish task completion events to the journal table. by @yruslan in #154
• #151 Specify number of threads to consume by a task. by @jirifilip in #156
• #149 Autodetect Enceladus info version by @yruslan in #155
• #162 Standardization sink by @yruslan in #163
• #167 Make checking of the workflow configuration file existence smarter. by @yruslan in #168
• #170 Delete empty folder on write error so readers won't be confused that the partition exists. by @yruslan in #172
• #164 Allow creation of Hive tables from StandardizationSink by @yruslan in #171
• #173 Unify JDBC retries config by @yruslan in #174

Full Changelog: v1.2.1...v1.2.2