Skip to content

Commit

Permalink
[SPARK-25883][SQL][MINOR] Override method prettyName in from_avro
Browse files Browse the repository at this point in the history
…/`to_avro`

Previously in from_avro/to_avro, we override the method `simpleString` and `sql` for the string output. However, the override only affects the alias naming:
```
Project [from_avro('col,
...
, (mode,PERMISSIVE)) AS from_avro(col, struct<col1:bigint,col2:double>, Map(mode -> PERMISSIVE))#11]
```
It only makes the alias name quite long: `from_avro(col, struct<col1:bigint,col2:double>, Map(mode -> PERMISSIVE))`).

We should follow `from_csv`/`from_json` here, to override the method prettyName only, and we will get a clean alias name

```
... AS from_avro(col)#11
```

Manual check

Closes #22890 from gengliangwang/revise_from_to_avro.

Authored-by: Gengliang Wang <gengliang.wang@databricks.com>
Signed-off-by: gatorsmile <gatorsmile@gmail.com>
  • Loading branch information
gengliangwang committed Nov 15, 2018
1 parent aaa21d8 commit 3323fdb
Show file tree
Hide file tree
Showing 2 changed files with 2 additions and 14 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -51,13 +51,7 @@ case class AvroDataToCatalyst(child: Expression, jsonFormatSchema: String)
deserializer.deserialize(result)
}

override def simpleString: String = {
s"from_avro(${child.sql}, ${dataType.simpleString})"
}

override def sql: String = {
s"from_avro(${child.sql}, ${dataType.catalogString})"
}
override def prettyName: String = "from_avro"

override protected def doGenCode(ctx: CodegenContext, ev: ExprCode): ExprCode = {
val expr = ctx.addReferenceObj("this", this)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -52,13 +52,7 @@ case class CatalystDataToAvro(child: Expression) extends UnaryExpression {
out.toByteArray
}

override def simpleString: String = {
s"to_avro(${child.sql}, ${child.dataType.simpleString})"
}

override def sql: String = {
s"to_avro(${child.sql}, ${child.dataType.catalogString})"
}
override def prettyName: String = "to_avro"

override protected def doGenCode(ctx: CodegenContext, ev: ExprCode): ExprCode = {
val expr = ctx.addReferenceObj("this", this)
Expand Down

0 comments on commit 3323fdb

Please sign in to comment.