Skip to content

Commit

Permalink
[SPARK-48474][CORE] Fix the class name of the log in `SparkSubmitArgu…
Browse files Browse the repository at this point in the history
…ments` & `SparkSubmit`

### What changes were proposed in this pull request?
The pr aims to fix `the class name` of the log in `SparkSubmitArguments` & `SparkSubmit`.

### Why are the changes needed?
We should display the class names that `match our understanding` in the logs, rather than the `anonymous class names` automatically generated by Scala.

```
sh bin/spark-shell --verbose
```

Before:
<img width="835" alt="image" src="https://github.com/apache/spark/assets/15246973/30843cb5-7a0e-4ec9-998a-36e9cbe09c33">

After:
<img width="840" alt="image" src="https://github.com/apache/spark/assets/15246973/94c526c7-3409-471a-bb89-b9c08f2a8d2d">

### Does this PR introduce _any_ user-facing change?
Yes, only for log.

### How was this patch tested?
Manually test.

### Was this patch authored or co-authored using generative AI tooling?
No.

Closes apache#46808 from panbingkun/SPARK-48474.

Authored-by: panbingkun <panbingkun@baidu.com>
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
  • Loading branch information
panbingkun authored and HyukjinKwon committed May 31, 2024
1 parent f64646c commit 80addbb
Show file tree
Hide file tree
Showing 2 changed files with 4 additions and 0 deletions.
2 changes: 2 additions & 0 deletions core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,8 @@ private[deploy] object SparkSubmitAction extends Enumeration {
*/
private[spark] class SparkSubmit extends Logging {

override protected def logName: String = classOf[SparkSubmit].getName

import DependencyUtils._
import SparkSubmit._

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,8 @@ private[deploy] class SparkSubmitArguments(args: Seq[String], env: Map[String, S
var submissionToRequestStatusFor: String = null
var useRest: Boolean = false // used internally

override protected def logName: String = classOf[SparkSubmitArguments].getName

/** Default properties present in the currently defined defaults file. */
lazy val defaultSparkProperties: HashMap[String, String] = {
val defaultProperties = new HashMap[String, String]()
Expand Down

0 comments on commit 80addbb

Please sign in to comment.