-
-
Notifications
You must be signed in to change notification settings - Fork 2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
perf: Make dt.truncate
1.5x faster when every
is just a single duration (and not an expression)
#16666
perf: Make dt.truncate
1.5x faster when every
is just a single duration (and not an expression)
#16666
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #16666 +/- ##
==========================================
- Coverage 81.51% 81.49% -0.02%
==========================================
Files 1414 1415 +1
Lines 186392 186609 +217
Branches 3014 3014
==========================================
+ Hits 151942 152083 +141
- Misses 33921 33996 +75
- Partials 529 530 +1 ☔ View full report in Codecov by Sentry. |
truncate
performance when every
is just a single duration (and not an expression)truncate
1.5x faster when every
is just a single duration (and not an expression)
…ration (and not an expression)
// TODO: optimize the code below, so it does the following: | ||
// - convert to naive | ||
// - truncate all naively | ||
// - localize, preserving the fold of the original datetime. | ||
// The last step is the non-trivial one. But it should be worth it, | ||
// and faster than the current approach of truncating everything | ||
// as tz-aware. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i've made an issue for this #16617
5425011
to
33b7347
Compare
What does the seperate implementation do that we cannot do in the |
In the broadcast one, the function:
If let every =
*duration_cache.get_or_insert_with(every, |every| Duration::parse(every));
if every.negative {
polars_bail!(ComputeError: "cannot truncate a Datetime to a negative duration")
} The cache would avoid the cost of If I think that the |
Yes.. Thanks for the explanation. 👍 |
truncate
1.5x faster when every
is just a single duration (and not an expression)truncate
1.5x faster when every
is just a single duration (and not an expression)
truncate
1.5x faster when every
is just a single duration (and not an expression)dt.truncate
1.5x faster when every
is just a single duration (and not an expression)
broadcast_try_binary_elementwise
simplifies things here, but does seem to introduce a perf hit for the base case (whenevery
isn't an expression): #15768 (comment)this makes a noticeable difference for the single-
every
case https://www.kaggle.com/code/marcogorelli/polars-timing?scriptVersionId=181122731: