Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding Constant Check for FilterExec #9649

Merged
merged 9 commits into from
Mar 18, 2024
Merged

Conversation

Lordworms
Copy link
Contributor

Which issue does this PR close?

Closes #9612

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

@github-actions github-actions bot added core Core DataFusion crate sqllogictest SQL Logic Tests (.slt) labels Mar 17, 2024
@Lordworms
Copy link
Contributor Author

Lordworms commented Mar 18, 2024

A problem here is that when dealing with c in ('a', 'b', 'c') the parser would translate it into a BinaryOp::Eq and we could not directly do add the left part to the constant. So I made it a special case.

@Lordworms Lordworms changed the title fix Bugs in adding SortExec for sorted tables Adding Constant Check for FilterExec Mar 18, 2024
@Lordworms Lordworms marked this pull request as ready for review March 18, 2024 02:21
@@ -131,7 +131,7 @@ fn update_coalesce_ctx_children(
coalesce_context.data = if children.is_empty() {
// Plan has no children, it cannot be a `CoalescePartitionsExec`.
false
} else if is_coalesce_partitions(&coalesce_context.plan) {
} else if is_coalesce_partitions(&coalesce_context.plan) || coalesce_context.data {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think none of the changes in this file is necessary for this PR. I retracted changes in this file. All tests still pass

Lordworms and others added 2 commits March 18, 2024 08:05
Co-authored-by: Mustafa Akur <106137913+mustafasrepo@users.noreply.github.com>
@Lordworms Lordworms requested a review from mustafasrepo March 18, 2024 13:11
Comment on lines 165 to 176
let data_vec: Vec<_> = adjusted.children.iter().map(|x| x.data).collect();
let mut plan_with_coalesce_partitions =
PlanWithCorrespondingCoalescePartitions::new_default(adjusted.plan);
plan_with_coalesce_partitions.children = plan_with_coalesce_partitions
.children
.into_iter()
.enumerate()
.map(|(id, mut child)| {
child.data = data_vec[id];
child
})
.collect();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think these changes are still unnecessary for this PR. Other than this, this PR is ready to merge. Thanks @Lordworms for this Pr.

@github-actions github-actions bot removed the core Core DataFusion crate label Mar 18, 2024
Copy link
Contributor

@mustafasrepo mustafasrepo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks @Lordworms again for this PR.

@Lordworms
Copy link
Contributor Author

Don't know why the win test failed, is this due to CI/CD issues?

@mustafasrepo
Copy link
Contributor

Don't know why the win test failed, is this due to CI/CD issues?

This unrelated to this Pr. There is an issue with windows runner. See issue: 9653

@mustafasrepo
Copy link
Contributor

Don't know why the win test failed, is this due to CI/CD issues?

This unrelated to this Pr. There is an issue with windows runner. See issue: 9653. This test is undeterministic (sometimes fails, sometimes passes). After CI completes I will re-rerun the test.

@Lordworms
Copy link
Contributor Author

Don't know why the win test failed, is this due to CI/CD issues?

This unrelated to this Pr. There is an issue with windows runner. See issue: 9653

Got it, thanks

@mustafasrepo mustafasrepo merged commit eb13f59 into apache:main Mar 18, 2024
23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
sqllogictest SQL Logic Tests (.slt)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Eliminate constant columns from sort requirements
2 participants