Add window frame constructs #492
Conversation
Codecov Report
@@ Coverage Diff @@
## master #492 +/- ##
==========================================
+ Coverage 75.95% 76.00% +0.05%
==========================================
Files 154 155 +1
Lines 26349 26494 +145
==========================================
+ Hits 20014 20138 +124
- Misses 6335 6356 +21
Continue to review full report at Codecov.
|
jimexist
force-pushed
the
add-window-frame
branch
3 times, most recently
from
b3da052
to
023509d
Compare
jimexist
force-pushed
the
add-window-frame
branch
from
023509d
to
0b5db2a
Compare
jimexist
force-pushed
the
add-window-frame
branch
from
0b5db2a
to
1db64de
Compare
|
Checking this one out... |
There was a problem hiding this comment.
I think this is looking good -- thanks @jimexist . I have some comments about code structure -- let me know what you think
Also, thank you very much for breaking this work up into multiple PRs @jimexist -- it really makes reviewing them much more tractable (though I realize it is more work for you).
| @@ -35,7 +35,7 @@ futures = "0.3" | |||
| log = "0.4" | |||
| prost = "0.7" | |||
| serde = {version = "1", features = ["derive"]} | |||
| sqlparser = "0.8" | |||
| //! - An EXCLUDE clause. | ||
|
|
||
| use crate::error::{DataFusionError, Result}; | ||
| use sqlparser::ast::{WindowFrame, WindowFrameBound, WindowFrameUnits}; |
There was a problem hiding this comment.
We can always change it later, but it might make sense eventually to translate from the SQL parser AST to datafusion specific structs. This would involve some code duplication, but might be helpful if we wanted to add DataFusion specific optimizations / annotations to the window. Just a thought for later. Not needed for this PR
There was a problem hiding this comment.
i see your point, let's review #506 as an alternative
| end_bound: Some(WindowFrameBound::CurrentRow), | ||
| }; | ||
|
|
||
| fn get_bound_rank(bound: &WindowFrameBound) -> (u8, u64) { |
There was a problem hiding this comment.
I don't understand the hard coded numbers in this function. Perhaps giving them names or adding some doc comments would help
| } | ||
|
|
||
| /// Validate a window frame if present, otherwise return the default window frame. | ||
| pub(crate) fn validate_window_frame( |
There was a problem hiding this comment.
In general, it would be cool if this validation was done as part of the SQL AST --> LogicalPlan / Expr. Then the rest of the optimizers could assume that the structures were valid.
if we added a DataFusion specific WindowFrame struct and then performed this validation while constructing it, by the time the physical plan saw a Window Frame it would be in a known good state
| let sql = "SELECT order_id, MAX(qty) OVER (ORDER BY order_id RANGE BETWEEN 3 PRECEDING and 3 FOLLOWING), MIN(qty) OVER (ORDER BY order_id DESC) from orders"; | ||
| let expected = "\ | ||
| Projection: #order_id, #MAX(qty), #MIN(qty)\ | ||
| \n WindowAggr: windowExpr=[[MAX(#qty)]] partitionBy=[]\ |
There was a problem hiding this comment.
I think it would be a good idea to have the window bounds appear in the explain plans
Which issue does this PR close?
Related #361
Based on #463
Rationale for this change
What changes are included in this PR?
Are there any user-facing changes?