Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Fix incorrect debug assertion in ChunkedArray::from_chunks_and_dtype #16697

Merged
merged 2 commits into from
Jun 4, 2024

Conversation

stinodego
Copy link
Member

@stinodego stinodego commented Jun 3, 2024

Converting an empty Array-of-Dates type resulted in a panic due to a debug assert:

pl.Series(dtype=pl.Array(pl.Date, 1)).to_list()
pyo3_runtime.PanicException: assertion `left == right` failed
  left: Date32
 right: Int32

Looks like empty arrays end up with the wrong data type here. I followed the trace but couldn't work out exactly why that happens. I did find this TODO comment which seems related:

// TODO: figure out how to deal with physical/logical distinction
// physical primitives like time, date etc. work
// physical nested need more

For now I have disabled the debug assertion for empty arrays. It resolves this issue.

@github-actions github-actions bot added internal An internal refactor or improvement python Related to Python Polars rust Related to Rust Polars labels Jun 3, 2024
@stinodego stinodego marked this pull request as ready for review June 3, 2024 22:13
Copy link

codecov bot commented Jun 3, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 81.48%. Comparing base (9912af0) to head (30635d8).
Report is 10 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #16697      +/-   ##
==========================================
- Coverage   81.49%   81.48%   -0.01%     
==========================================
  Files        1416     1416              
  Lines      186880   186880              
  Branches     3023     3023              
==========================================
- Hits       152289   152279      -10     
- Misses      34059    34070      +11     
+ Partials      532      531       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@stinodego stinodego marked this pull request as draft June 4, 2024 06:02
@@ -206,8 +206,8 @@ where
// that check if the data types in the arrays are as expected
#[cfg(debug_assertions)]
{
if !chunks.is_empty() && dtype.is_primitive() {
assert_eq!(chunks[0].data_type(), &dtype.to_physical().to_arrow(true))
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The .to_physical() here is unneeded as we have already checked that dtype is primitive.

@stinodego stinodego marked this pull request as ready for review June 4, 2024 06:28
Copy link

codspeed-hq bot commented Jun 4, 2024

CodSpeed Performance Report

Merging #16697 will not alter performance

Comparing hyp-array (30635d8) with main (9912af0)

Summary

✅ 37 untouched benchmarks

@ritchie46 ritchie46 merged commit d15a376 into main Jun 4, 2024
26 checks passed
@ritchie46 ritchie46 deleted the hyp-array branch June 4, 2024 09:15
@c-peters c-peters added the accepted Ready for implementation label Jun 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
accepted Ready for implementation internal An internal refactor or improvement python Related to Python Polars rust Related to Rust Polars
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

3 participants