You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Sometime a table can be late as of BAU. And Pramen can pick it up and run the corresponding transformation as soon as the table loads. But before that Pramen currently considers it a failure.
Feature
Allow specifying for a transformation that it can be late (e.g. dependency checks fail), skip it, but not trigger the pipeline failure.
yruslan
changed the title
Allow to ignore late data for specified tables or transformations
Allow to ignore late data for specified transformations
Jan 12, 2023
Background
Sometime a table can be late as of BAU. And Pramen can pick it up and run the corresponding transformation as soon as the table loads. But before that Pramen currently considers it a failure.
Feature
Allow specifying for a transformation that it can be late (e.g. dependency checks fail), skip it, but not trigger the pipeline failure.
Example
For a transformation dependency
Update. Found a good name for a dependency that is not optional, but does not raise errors if the job is skipped because of it - passive.
The text was updated successfully, but these errors were encountered: