Skip to content

Commit

Permalink
[SPARK-44816][CONNECT] Improve error message when UDF class is not found
Browse files Browse the repository at this point in the history
### What changes were proposed in this pull request?

Improve the error messaging on the connect client when using
a UDF whose corresponding class has not been sync'ed with the
spark connect service.

Prior to this change, the client receives a cryptic error:

```
Exception in thread "main" org.apache.spark.SparkException: Main$
```

With this change, the message is improved to be:

```
Exception in thread "main" org.apache.spark.SparkException: Failed to load class: Main$. Make sure the artifact where the class is defined is installed by calling session.addArtifact.
```

### Why are the changes needed?

This change makes it clear to the user on what the error is.

### Does this PR introduce _any_ user-facing change?

Yes. The error message is improved. See details above.

### How was this patch tested?

Manually by running a connect server and client.

Closes apache#42500 from nija-at/improve-error.

Authored-by: Niranjan Jayakar <nija@databricks.com>
Signed-off-by: Herman van Hovell <herman@databricks.com>
  • Loading branch information
nija-at authored and hvanhovell committed Aug 23, 2023
1 parent dd6cda5 commit 2d0a0a0
Showing 1 changed file with 5 additions and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -1523,6 +1523,11 @@ class SparkConnectPlanner(val sessionHolder: SessionHolder) extends Logging {
s"Failed to load class correctly due to $nsm. " +
"Make sure the artifact where the class is defined is installed by calling" +
" session.addArtifact.")
case cnf: ClassNotFoundException =>
throw new ClassNotFoundException(
s"Failed to load class: ${cnf.getMessage}. " +
"Make sure the artifact where the class is defined is installed by calling" +
" session.addArtifact.")
case _ => throw t
}
}
Expand Down

0 comments on commit 2d0a0a0

Please sign in to comment.