Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Raise informative error on Unknown unnest #19830

Merged
merged 2 commits into from
Nov 17, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 37 additions & 2 deletions crates/polars-plan/src/plans/conversion/expr_expansion.rs
Original file line number Diff line number Diff line change
Expand Up @@ -337,8 +337,43 @@ fn expand_struct_fields(
unreachable!()
};
let field = input[0].to_field(schema, Context::Default)?;
let DataType::Struct(fields) = field.dtype() else {
polars_bail!(InvalidOperation: "expected 'struct'")
let dtype = field.dtype();
let DataType::Struct(fields) = dtype else {
if !dtype.is_known() {
let mut msg = String::from(
"expected 'struct' got an unknown data type

This means there was an operation of which the output data type could not be determined statically.
Try setting the output data type for that operation.",
);
for e in input[0].into_iter() {
#[allow(clippy::single_match)]
match e {
#[cfg(feature = "list_to_struct")]
Expr::Function {
input: _,
function,
options: _,
} => {
if matches!(
function,
FunctionExpr::ListExpr(ListFunction::ToStruct(..))
) {
msg.push_str(
"

Hint: set 'upper_bound' for 'list.to_struct'.",
);
}
},
_ => {},
}
}

polars_bail!(InvalidOperation: msg)
} else {
polars_bail!(InvalidOperation: "expected 'struct' got {}", field.dtype())
}
};

// Wildcard.
Expand Down
5 changes: 5 additions & 0 deletions py-polars/polars/expr/list.py
Original file line number Diff line number Diff line change
Expand Up @@ -1125,6 +1125,11 @@ def to_struct(

Notes
-----
It is recommended to set 'upper_bound' to the correct output size of the struct.
If this is not set, Polars will not know the output type of this operation and
will set it to 'Unknown' which can lead to errors because Polars is not able
to resolve the query.

For performance reasons, the length of the first non-null sublist is used
to determine the number of output fields. If the sublists can be of different
lengths then `n_field_strategy="max_width"` must be used to obtain the expected
Expand Down