-
Notifications
You must be signed in to change notification settings - Fork 425
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
TEZ-4526: Avoid calling LocationProvider#getPreferredLocations multiple times while generating grouped splits #323
Conversation
This comment was marked as outdated.
This comment was marked as outdated.
Requesting @rbalamohan @abstractdog for reviews. |
…le times while generating grouped splits
a1137a6
to
34409ed
Compare
💔 -1 overall
This message was automatically generated. |
Time taken for grouping splits - Generated 30000 file splits and generated 1000 grouped splits with each grouped split having 30 file splits.
I dont see a regression happening with the patch. I think I have addressed the regression comments on TEZ-4069. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, thanks for the perf testing!
…le times while generating grouped splits (#323)
…le times while generating grouped splits (#323) - CHANGES.txt
TEZ-4526: Avoid calling LocationProvider#getPreferredLocations multiple times while generating grouped splits