coalesce function returns incorrect result

[viirya]

Describe the bug

Currently coalesce function's return type is determined by a fixed list of SUPPORTED_COALESCE_TYPES. If the data type of input arguments are not in the list, DataFusion will coerce input arguments. It produces some weird results.

For example, if the input arguments are [Data32, Date32], because SUPPORTED_COALESCE_TYPES doesn't include Date32, the return type of coalesce function will be Utf8.

This doesn't look like correct based on the definition of coalesce function which should return first non-null value from the arguments. As the input arguments are same type, there is no need to apply a coerced type on them (Utf8).

coalesce function simply checks null bits from input arrays, and zip input arrays accordingly. DataFusion doesn't manipulate the arrays but relies on arrow-rs kernels like and, zip, etc. Looks like we don't need to limit the input types of coalesce to SUPPORTED_COALESCE_TYPES.

The current approach is strange especially it determines the return type before coercing the input arguments. So we can construct a coalesce function with [Data32, Date32] inputs and its return type is Utf8. It is especially an issue for projects like Comet which only uses DataFusion physical plans, so we don't pass through the type coercion phrase of logical plan in DataFusion. So in Comet, when we construct a coalesce function with [Data32, Date32] inputs, the function will actually produce a Date32 array but the schema is Utf8.

I believe when we construct coalesce function, we should make sure its return type match its input types.

To Reproduce

No response

Expected behavior

No response

Additional context

No response