You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Description
add dpl command for enforced batch/driver execution. currently aggregations are processed in paralel mode which does not permit all the possible use cases that spark offers.
Use case or motivation behind the feature request
there are cases where the aggregation should happen only in sequential mode and forget the previous batch. such are when only batch-window aggregations are required like for example supporting a use case where trigger is configured for 5 minutes and the data between now and the last 5 minutes would be required to do the aggregation.
Related issues
Additional context
The text was updated successfully, but these errors were encountered:
Description
add dpl command for enforced batch/driver execution. currently aggregations are processed in paralel mode which does not permit all the possible use cases that spark offers.
Use case or motivation behind the feature request
there are cases where the aggregation should happen only in sequential mode and forget the previous batch. such are when only batch-window aggregations are required like for example supporting a use case where trigger is configured for 5 minutes and the data between now and the last 5 minutes would be required to do the aggregation.
Related issues
Additional context
The text was updated successfully, but these errors were encountered: