Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: NDarray/Tensor support #16466

Merged
merged 9 commits into from
May 25, 2024
Merged

feat: NDarray/Tensor support #16466

merged 9 commits into from
May 25, 2024

Conversation

ritchie46
Copy link
Member

On the 1.0 release we will

  • change the constructor of Series to read 2D arrays to Array type instead of List.
  • Default reshape to Array instead of List.

Higher dimensions will now go to Array with a multidimensional shape instead of Object.

Constructors coming

>>> pl.Series(range(64)).reshape((4, 2, 1, -1), nested_type=pl.Array)
shape: (4,)
Series: '' [array[i64, shape: (2, 1, 8)]]
[
    [[[0, 1, … 7]], [[8, 9, … 15]]]
    [[[16, 17, … 23]], [[24, 25, … 31]]]
    [[[32, 33, … 39]], [[40, 41, … 47]]]
    [[[48, 49, … 55]], [[56, 57, … 63]]]
]

@github-actions github-actions bot added enhancement New feature or an improvement of an existing feature python Related to Python Polars rust Related to Rust Polars labels May 24, 2024
@stinodego stinodego modified the milestone: 1.0.0 May 24, 2024
Copy link

codecov bot commented May 24, 2024

Codecov Report

Attention: Patch coverage is 69.31217% with 58 lines in your changes are missing coverage. Please review.

Project coverage is 81.43%. Comparing base (717277e) to head (46545fa).
Report is 15 commits behind head on main.

Files Patch % Lines
crates/polars-ops/src/series/ops/reshape.rs 58.53% 34 Missing ⚠️
crates/polars-plan/src/dsl/function_expr/schema.rs 37.50% 5 Missing ⚠️
crates/polars-core/src/datatypes/dtype.rs 84.61% 4 Missing ⚠️
py-polars/polars/datatypes/classes.py 82.60% 2 Missing and 2 partials ⚠️
crates/polars-plan/src/dsl/function_expr/mod.rs 25.00% 3 Missing ⚠️
...lars/polars/testing/parametric/strategies/dtype.py 50.00% 2 Missing ⚠️
crates/polars-expr/src/expressions/mod.rs 0.00% 1 Missing ⚠️
...ates/polars-plan/src/dsl/function_expr/dispatch.rs 75.00% 1 Missing ⚠️
crates/polars-plan/src/dsl/function_expr/list.rs 50.00% 1 Missing ⚠️
crates/polars-plan/src/dsl/options.rs 0.00% 1 Missing ⚠️
... and 2 more
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #16466      +/-   ##
==========================================
- Coverage   81.45%   81.43%   -0.03%     
==========================================
  Files        1409     1409              
  Lines      184713   184946     +233     
  Branches     2971     2978       +7     
==========================================
+ Hits       150467   150614     +147     
- Misses      33726    33811      +85     
- Partials      520      521       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@@ -510,15 +511,17 @@ def numpy_to_pyseries(
strict=strict,
nan_to_null=nan_to_null,
)
return (
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We definitely don't want to go through the engine during construction. So added reshape directly on Series.

@@ -500,6 +499,8 @@ def numpy_to_pyseries(
return constructor(
name, values, nan_to_null if dtype in (np.float32, np.float64) else strict
)
# TODO: remove this branch on 1.0.
# This returns a List whereas we should return an Array type
elif values.ndim == 2:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This branch still returns List<i64> for 2D arrays. Will remove that during 1.0, so we return an Array type here.

@ritchie46 ritchie46 merged commit 019dfe8 into main May 25, 2024
27 checks passed
@ritchie46 ritchie46 deleted the tensor branch May 25, 2024 08:21
@c-peters c-peters added the accepted Ready for implementation label May 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
accepted Ready for implementation enhancement New feature or an improvement of an existing feature python Related to Python Polars rust Related to Rust Polars
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

3 participants