Skip to content

Commit

Permalink
[SPARK-40984][CORE][SQL] Use NON_FOLDABLE_INPUT instead of `FRAME_L…
Browse files Browse the repository at this point in the history
…ESS_OFFSET_WITHOUT_FOLDABLE`

### What changes were proposed in this pull request?
This pr aims replace `FRAME_LESS_OFFSET_WITHOUT_FOLDABLE` with `NON_FOLDABLE_INPUT` to clean up similar error subclass.

### Why are the changes needed?
`FRAME_LESS_OFFSET_WITHOUT_FOLDABLE` and `NON_FOLDABLE_INPUT` look similar, but `NON_FOLDABLE_INPUT` is more general

### Does this PR introduce _any_ user-facing change?
Yes. The PR changes user-facing error messages.

For example, the error message will change from

```
Offset expression "(- nonfoldableliteral())\" must be a literal.
```

to

```
the input offset should be a foldable "INT" expression; however, got "(- nonfoldableliteral())\".
```

### How was this patch tested?
Pass Github Actions

Closes #38536 from LuciferYang/SPARK-40984.

Authored-by: yangjie01 <yangjie01@baidu.com>
Signed-off-by: Max Gekk <max.gekk@gmail.com>
  • Loading branch information
LuciferYang authored and MaxGekk committed Nov 8, 2022
1 parent 755ed45 commit 5c6c5e6
Show file tree
Hide file tree
Showing 4 changed files with 15 additions and 11 deletions.
5 changes: 0 additions & 5 deletions core/src/main/resources/error/error-classes.json
Original file line number Diff line number Diff line change
Expand Up @@ -209,11 +209,6 @@
"Input to <functionName> should all be the same type, but it's <dataType>."
]
},
"FRAME_LESS_OFFSET_WITHOUT_FOLDABLE" : {
"message" : [
"Offset expression <offset> must be a literal."
]
},
"HASH_MAP_TYPE" : {
"message" : [
"Input to the function <functionName> cannot contain elements of the \"MAP\" type. In Spark, same maps may have different hashcode, thus hash expressions are prohibited on \"MAP\" elements. To restore previous behavior set \"spark.sql.legacy.allowHashOnMapType\" to \"true\"."
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -452,9 +452,11 @@ sealed abstract class FrameLessOffsetWindowFunction
check
} else if (!offset.foldable) {
DataTypeMismatch(
errorSubClass = "FRAME_LESS_OFFSET_WITHOUT_FOLDABLE",
errorSubClass = "NON_FOLDABLE_INPUT",
messageParameters = Map(
"offset" -> toSQLExpr(offset)
"inputName" -> "offset",
"inputType" -> toSQLType(offset.dataType),
"inputExpr" -> toSQLExpr(offset)
)
)
} else {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -770,8 +770,12 @@ class ExpressionTypeCheckingSuite extends SparkFunSuite with SQLHelper with Quer
val lag = Lag(Literal(1), NonFoldableLiteral(10), Literal(null), true)
assert(lag.checkInputDataTypes() ==
DataTypeMismatch(
errorSubClass = "FRAME_LESS_OFFSET_WITHOUT_FOLDABLE",
messageParameters = Map("offset" -> "\"(- nonfoldableliteral())\"")
errorSubClass = "NON_FOLDABLE_INPUT",
messageParameters = Map(
"inputName" -> "offset",
"inputType" -> "\"INT\"",
"inputExpr" -> "\"(- nonfoldableliteral())\""
)
))
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -881,10 +881,13 @@ class DataFrameWindowFunctionsSuite extends QueryTest
lag($"value", 3, null, true).over(window),
lag(concat($"value", $"key"), 1, null, true).over(window)).orderBy($"order").collect()
},
errorClass = "DATATYPE_MISMATCH.FRAME_LESS_OFFSET_WITHOUT_FOLDABLE",
errorClass = "DATATYPE_MISMATCH.NON_FOLDABLE_INPUT",
parameters = Map(
"sqlExpr" -> "\"lag(value, nonfoldableliteral(), NULL)\"",
"offset" -> "\"(- nonfoldableliteral())\"")
"inputName" -> "offset",
"inputType" -> "\"INT\"",
"inputExpr" -> "\"(- nonfoldableliteral())\""
)
)
}

Expand Down

0 comments on commit 5c6c5e6

Please sign in to comment.