Relaxing the type check when creating WindowFunction

[JkSelf]

While executing Gluten, we encountered the following error message:

Error Source: USER
Error Code: INVALID_ARGUMENT
Reason: Unexpected return type for window function sum(REAL). Expected REAL. Got DOUBLE.
Retriable: False

The reason for this issue is that when the window function is created, it verifies whether the return value of the registered sum(REAL) function is of REAL type. If the type passed in is double, it will trigger an error. However, in actual operation, converting a real type to a double type does not affect the outcome. Therefore, could we consider relaxing the type check here? For real and double types, we might not need to enforce such strict matching checks. Would this approach be feasible?

[JkSelf]

@mbasmanova Do you have any input? Thanks

[mbasmanova]

@JkSelf Typically this flexibility is achieved by introducing type casts. The query planner needs to explicitly cast the result of sum(real) to double.

[JkSelf]

@mbasmanova After discussing with @rui-mo offline, it seems we could introduce a function registration for sum(real) to return a double. This approach might reduce the overhead associated with type casting. What are your thoughts on this?

[mbasmanova]

@JkSelf There is not enough context here. To make progress, I suggest to take the following steps and share the findings.

• Come up with a simplest SQL query that reproduces the problem.
• Examine Spark's physical plan for the query.
• Examine translation from Spark's plan to Velox plan.
• Double check SUM function's signature in Spark and verify that Velox matches. Is it sum(real) -> real or sum(real) -> double. Note that it cannot be both.

Wild guess: perhaps, there is a bug in Gluten where it uses 'sum' function, but should be using 'sum_partial'.

[JkSelf]

@mbasmanova This is a bug in Velox related to registering the Spark aggregate function. In Spark SQL, the return type of sum(real) should be double, not real here. However, in Velox, it is incorrectly set as real. I have addressed this issue in this PR. Could you please review it? Thank you.

[JkSelf]

Fixed in #9818. Will close this discussion. Thanks @mbasmanova.