[SPARK-3869] ./bin/spark-class miss Java version with _JAVA_OPTIONS set

[cocoatomo]

When _JAVA_OPTIONS environment variable is set, a command "java -version" outputs a message like "Picked up _JAVA_OPTIONS: -Dfile.encoding=UTF-8".
./bin/spark-class knows java version from the first line of "java -version" output, so it mistakes java version with _JAVA_OPTIONS set.

[AmplabJenkins]

Can one of the admins verify this patch?

[AmplabJenkins]

Can one of the admins verify this patch?

[JoshRosen]

Jenkins, add to whitelist.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21677/
Test FAILed.

[SparkQA]

QA tests have started for PR 2725 at commit f894ebd.

• This patch merges cleanly.

[SparkQA]

QA tests have finished for PR 2725 at commit f894ebd.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[andrewor14]

Hey @cocoatomo I might be missing something. What do you mean by "_JAVA_OPTIONS set"? How does your fix cause that output to be printed?

[vanzin]

Funny. Seems to be a feature of the JVM itself, but I can't find any documentation about it.

$ _JAVA_OPTIONS=-Dfoo=bar java -version
Picked up _JAVA_OPTIONS: -Dfoo=bar
java version "1.6.0_37"
Java(TM) SE Runtime Environment (build 1.6.0_37-b06)
Java HotSpot(TM) 64-Bit Server VM (build 20.12-b01, mixed mode)

[andrewor14]

Ahh I see. That is very strange indeed. I think in general it's a good idea to grep for version anyway. Thanks I'm merging this.

[cocoatomo]

Hi, @andrewor14 @vanzin . Thank you for your comments.

A value of _JAVA_OPTIONS is automatically passed to java command as an argument.
Because the default file.encoding on Mac OS X is MacRoman which is not confortable,
it is common to set _JAVA_OPTIONS=-Dfile.encoding=UTF-8.

Thanks!