Add spark_application_id to journal

[yruslan]

Background

This would help generating reports based on journal table.

Feature

Add spark_application_id to journal.

Example [Optional]

--

Proposed Solution [Optional]

Also add:

• Pipeline ID (generate on fly)
• Pipeline name
• Environment
• Tenant

[VladimirRybalko]

I'll take this one and also prepare the concept of bookkeeping.
What we want to achieve looks very similar to the proper bookkeeping pattern being used for monitoring and compliance purpose.

[yruslan]

Sure!

I didn't add much context to this issue, please reach me out if you have questions.