Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[logical-types] use Scalar in Expr::Logical #12793

Merged
merged 4 commits into from
Oct 9, 2024

Conversation

notfilippo
Copy link
Contributor

Item for #12622. (See plan described in the EPIC for context around this change)

@github-actions github-actions bot added sql SQL Planner logical-expr Logical plan and expressions physical-expr Physical Expressions optimizer Optimizer rules core Core DataFusion crate sqllogictest SQL Logic Tests (.slt) substrait proto Related to proto crate functions labels Oct 7, 2024
pub struct Scalar {
value: ScalarValue,
pub value: ScalarValue,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've discovered just now that it's possible to match inside a partially public struct. We could consider removing the value and into_value method and just use the field.

Not all matches were update to support pattern-matching this field (I've discovered this feature in the middle of this changes). They can be updated subsequently.


impl fmt::Debug for Scalar {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
self.value.fmt(f)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please keep derived Debug. It reports both value & type, which might be useful for debugging.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is me being lazy since some tests were failing because they compare debug prints of plans. I can revert this, it was added to not introduce other changes.

@@ -116,6 +143,10 @@ impl Scalar {
ScalarValue::iter_to_array_of_type(scalars.map(|scalar| scalar.value), &data_type)
}

pub fn cast_to(&self, data_type: &DataType) -> Result<Self> {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need this on the Scalar itself?
Once we logicalize it, this function will drag us. Maybe let's have a utlity method somewhere else.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this function is useful to keep here because we can then enforce the same constraints introduced in the construction of the struct and potentially support logic like

data_type.is_logically_equivalent_to(self.data_type) => self.data_type = data_type (skipping the cast)

Comment on lines -384 to -388
let utf8_val = if utf8_val == "foo" {
"bar".to_string()
} else {
utf8_val
};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: avoid formatting changes unless needed

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Weird. I just ran cargo fmt. 🤷

@@ -41,39 +42,45 @@ pub trait TimestampLiteral {
fn lit_timestamp_nano(&self) -> Expr;
}

impl From<ScalarValue> for Expr {
fn from(value: ScalarValue) -> Self {
Self::Literal(Scalar::from(value))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

at some point we should rename Literal to Constant. constant is what this represents, while literal is a syntax-related term.

@alamb
Copy link
Contributor

alamb commented Oct 9, 2024

Let's keep this code moving on to the branch

Thank you for your review @findepi

@alamb alamb merged commit 22cb506 into apache:logical-types Oct 9, 2024
26 checks passed
@notfilippo notfilippo deleted the fr/scalar-in-expr branch October 10, 2024 08:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Core DataFusion crate functions logical-expr Logical plan and expressions optimizer Optimizer rules physical-expr Physical Expressions proto Related to proto crate sql SQL Planner sqllogictest SQL Logic Tests (.slt) substrait
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants