Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ARROW-11259: [Python] Allow to create field reference to nested field #12863

Closed
wants to merge 6 commits into from

Conversation

AlenkaF
Copy link
Member

@AlenkaF AlenkaF commented Apr 12, 2022

@github-actions
Copy link

@github-actions
Copy link

⚠️ Ticket has not been started in JIRA, please click 'Start Progress'.

Copy link
Member

@lidavidm lidavidm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you! I left a couple thoughts.

python/pyarrow/tests/test_dataset.py Outdated Show resolved Hide resolved
python/pyarrow/compute.py Outdated Show resolved Hide resolved
python/pyarrow/_compute.pyx Show resolved Hide resolved
@@ -1376,12 +1376,18 @@ TEST(Expression, SerializationRoundTrips) {

ExpectRoundTrips(field_ref("field"));

ExpectRoundTrips(field_ref(FieldRef("foo", "bar", "baz")));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have to spell out field_ref(FieldRef("foo", "bar")) explicitly or is it possible to simply write field_ref("foo", "bar")?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will try to simplify.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently it doesn't work, FieldRef("foo", "bar") needs to be spelled out.

Try 1: `field_ref("foo", "bar")`
```
/Users/alenkafrim/repos/arrow/cpp/src/arrow/compute/exec/expression_test.cc:1379:20: error: no matching function for call to 'field_ref'
  ExpectRoundTrips(field_ref("foo", "bar", "baz"));
                   ^~~~~~~~~
/Users/alenkafrim/repos/arrow/cpp/src/arrow/compute/exec/expression.h:152:12: note: candidate function not viable: requires single argument 'ref', but 3 arguments were provided
Expression field_ref(FieldRef ref);
```
Try 2: `field_ref(("foo", "bar"))`
```
/Users/alenkafrim/repos/arrow/cpp/src/arrow/compute/exec/expression_test.cc:1379:31: warning: expression result unused [-Wunused-value]
  ExpectRoundTrips(field_ref(("foo", "bar", "baz")));
```

python/pyarrow/compute.py Outdated Show resolved Hide resolved
@AlenkaF
Copy link
Member Author

AlenkaF commented Apr 14, 2022

Thank you for the comments @lidavidm @pitrou!
I think this PR is ready for another round of review (the errors should be related to the S3fs) .

Copy link
Member

@pitrou pitrou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update @AlenkaF ! Just two minor comments.

python/pyarrow/tests/test_dataset.py Show resolved Hide resolved
python/pyarrow/tests/test_dataset.py Outdated Show resolved Hide resolved
@AlenkaF AlenkaF requested a review from pitrou April 15, 2022 06:08
Copy link
Member

@pitrou pitrou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, will merge if green. Thank you @AlenkaF !

@pitrou pitrou closed this in e0668e2 Apr 18, 2022
@ursabot
Copy link

ursabot commented Apr 18, 2022

Benchmark runs are scheduled for baseline = 83dda5c and contender = e0668e2. e0668e2 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Failed ⬇️0.5% ⬆️0.04%] test-mac-arm
[Failed ⬇️0.0% ⬆️0.0%] ursa-i9-9960x
[Finished ⬇️0.26% ⬆️0.0%] ursa-thinkcentre-m75q
Buildkite builds:
[Finished] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/524| e0668e20 ec2-t3-xlarge-us-east-2>
[Failed] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/511| e0668e20 test-mac-arm>
[Failed] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/510| e0668e20 ursa-i9-9960x>
[Finished] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/521| e0668e20 ursa-thinkcentre-m75q>
[Finished] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/523| 83dda5c3 ec2-t3-xlarge-us-east-2>
[Failed] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/510| 83dda5c3 test-mac-arm>
[Failed] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/509| 83dda5c3 ursa-i9-9960x>
[Finished] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/520| 83dda5c3 ursa-thinkcentre-m75q>
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants