You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When using the arith kernel's is_null or is_not_null on arrays of type NullArray (with datatype Null) the .nulls() buffer is always empty for these types. Therefor the kernel returns an array with all values false/true which is the exact opposite of what should be returned.
fn is_null_op_allnull() -> Result<()> {
let schema = Schema::new(vec![Field::new("a", DataType::Null, true)]);
let mut a = NullBuilder::with_capacity(1);
// expression: "a is null"
let expr = is_null(col("a", &schema)?).unwrap();
let batch = RecordBatch::try_new(Arc::new(schema), vec![Arc::new(a.finish())])?;
let result = expr.evaluate(&batch)?.into_array(batch.num_rows());
let result =
as_boolean_array(&result).expect("failed to downcast to BooleanArray");
let expected = &BooleanArray::from(vec![true]);
assert_eq!(expected, result);
Ok(())
}
Expected behavior
for is_null I expect a BoolArray of true values for NullArray input (irrespective of length).
for is_not_null I expect a BoolArray of false values for NullArray input (irrespective of length).
Additional context
The text was updated successfully, but these errors were encountered:
I can create a pr for this with my solution but was curious if the NullArray::nulls should return a nullbuffer based on self length or if the kernel should be made Null aware
Describe the bug
When using the arith kernel's
is_null
oris_not_null
on arrays of type NullArray (with datatype Null) the .nulls() buffer is always empty for these types. Therefor the kernel returns an array with all values false/true which is the exact opposite of what should be returned.To Reproduce
Noticed it here apache/datafusion#7038
Run this unit test in
datafusion/is_null.rs
Expected behavior
for is_null I expect a BoolArray of true values for NullArray input (irrespective of length).
for is_not_null I expect a BoolArray of false values for NullArray input (irrespective of length).
Additional context
The text was updated successfully, but these errors were encountered: