Glassfish logs JVM options (including passwords) on startup

[dheles]

As appears to be its default behavior, Glassfish logs the JVM options to server.log on startup. Unfortunately, this includes doi and rserve passwords. I have been unable to find a way to avoid this.

Example log entry:
[2018-12-19T18:40:00.967+0000] [] [INFO] [NCLS-GFLAUNCHER-00005] [javax.enterprise.launcher] [tid: _ThreadID=1 _ThreadName=main] [timeMillis: 1545244800967] [levelValue: 800] [[ JVM invocation command line: ... -Ddataverse.rserve.password=[PASSWORD] -Ddoi.password=[PASSWORD]
glassfish code responsible:
https://github.com/eclipse-ee4j/glassfish/blob/glassfish-main-aggregator-5.1.0-RC1/nucleus/admin/launcher/src/main/java/com/sun/enterprise/admin/launcher/GFLauncherLogger.java#L159

The log level in question is only INFO, so I'd be happy to simply raise it to WARNING, but neither logging.properties nor asadmin list-log-levels seen to contain the relevant logger.

Alternately, moving the passwords to the database* would help.

• preferably encrypted

[pdurbin]

@dheles and I have been discussing this a bit at http://irclog.iq.harvard.edu/dataverse/2018-12-19#i_83268 a bit and I wanted to point out that printing the JVM options like this is the out of the box Glassfish behavior. In the example below, I installed Glassfish 4.1 fresh into /tmp/gf and ran asadmin start-domain to start it. Here's how the out of the box JVM options (nothing about Dataverse) look in server.log:

[2018-12-13T15:24:04.581+0000] [] [INFO] [NCLS-GFLAUNCHER-00005] [javax.enterprise.launcher] [tid: _ThreadID=1 _ThreadName=main] [timeMillis: 1544714644581] [levelValue: 800] [[
  JVM invocation command line:
/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.191.b12-1.el7_6.x86_64/jre/bin/java
-cp
/tmp/gf/glassfish4/glassfish/modules/glassfish.jar
-XX:+UnlockDiagnosticVMOptions
-XX:NewRatio=2
-XX:MaxPermSize=192m
-Xmx512m
-javaagent:/tmp/gf/glassfish4/glassfish/lib/monitor/flashlight-agent.jar
-client
-Djavax.xml.accessExternalSchema=all
-Djavax.net.ssl.trustStore=/tmp/gf/glassfish4/glassfish/domains/domain1/config/cacerts.jks
-Djdk.corba.allowOutputStreamSubclass=true
-Dfelix.fileinstall.dir=/tmp/gf/glassfish4/glassfish/modules/autostart/
-Dorg.glassfish.additionalOSGiBundlesToStart=org.apache.felix.shell,org.apache.felix.gogo.runtime,org.apache.felix.gogo.shell,org.apache.felix.gogo.command,org.apache.felix.shell.remote,org.apache.felix.fileinstall
-Dcom.sun.aas.installRoot=/tmp/gf/glassfish4/glassfish
-Dfelix.fileinstall.poll=5000
-Djava.endorsed.dirs=/tmp/gf/glassfish4/glassfish/modules/endorsed:/tmp/gf/glassfish4/glassfish/lib/endorsed
-Djava.security.policy=/tmp/gf/glassfish4/glassfish/domains/domain1/config/server.policy
-Dosgi.shell.telnet.maxconn=1
-Dfelix.fileinstall.bundles.startTransient=true
-Dcom.sun.enterprise.config.config_environment_factory_class=com.sun.enterprise.config.serverbeans.AppserverConfigEnvironmentFactory
-Dfelix.fileinstall.log.level=2
-Djavax.net.ssl.keyStore=/tmp/gf/glassfish4/glassfish/domains/domain1/config/keystore.jks
-Djava.security.auth.login.config=/tmp/gf/glassfish4/glassfish/domains/domain1/config/login.conf
-Dfelix.fileinstall.disableConfigSave=false
-Dfelix.fileinstall.bundles.new.start=true
-Dcom.sun.aas.instanceRoot=/tmp/gf/glassfish4/glassfish/domains/domain1
-Dosgi.shell.telnet.port=6666
-Dgosh.args=--nointeractive
-Dcom.sun.enterprise.security.httpsOutboundKeyAlias=s1as
-Dosgi.shell.telnet.ip=127.0.0.1
-DANTLR_USE_DIRECT_CLASS_LOADING=true
-Djava.awt.headless=true
-Dcom.ctc.wstx.returnNullForDefaultNamespace=true
-Djava.ext.dirs=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.191.b12-1.el7_6.x86_64/jre/lib/ext:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.191.b12-1.el7_6.x86_64/jre/jre/lib/ext:/tmp/gf/glassfish4/glassfish/domains/domain1/lib/ext
-Djdbc.drivers=org.apache.derby.jdbc.ClientDriver
-Djava.library.path=/tmp/gf/glassfish4/glassfish/lib:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
com.sun.enterprise.glassfish.bootstrap.ASMain
-upgrade
false
-domaindir
/tmp/gf/glassfish4/glassfish/domains/domain1
-read-stdin
true
-asadmin-args
--host,,,localhost,,,--port,,,4848,,,--secure=false,,,--terse=false,,,--echo=false,,,--interactive=true,,,start-domain,,,--verbose=false,,,--watchdog=false,,,--debug=false,,,--domaindir,,,/tmp/gf/glassfish4/glassfish/domains,,,domain1
-domainname
domain1
-instancename
server
-type
DAS
-verbose
false
-asadmin-classpath
/tmp/gf/glassfish4/glassfish/lib/client/appserver-cli.jar
-debug
false
-asadmin-classname
com.sun.enterprise.admin.cli.AdminMain]]

Over at https://javabot.evanchooly.com/logs/%23glassfish/2018-12-19 I asked if anyone knows if it's possible to suppress this output.

[pdurbin]

A very helpful person in #glassfish mentioned "password aliases" which lead me to the following quote at https://blog.eisele.net/2011/05/securing-your-glassfish-hardening-guide.html

"You should change your resource passwords to aliased ones. Use the asadmin create-password-alias cmd to change clear-text passwords in domain.xml to ${ALIAS=xxxx} entries."

It's hard to find HTML versions of Glassfish docs these days but here are some docs for Glassfish 3: https://docs.oracle.com/cd/E19798-01/821-1751/ghgqc/index.html

Payara has some HTML docs: https://docs.payara.fish/documentation/payara-server/password-aliases/

Anyway, from IRC sounds like @dheles is going to play around see what he can figure out what's possible without any code changes.

[poikilotherm]

Should I mention #5293 or not?

An example for the password aliases usage can be found here: https://github.com/poikilotherm/dataverse/blob/5292-small-container/conf/docker/app/init_2_configure.sh Beware, this is using Payara 5. Your mileage may vary.

[pdurbin]

@poikilotherm yes! I didn't know you are already playing with create-password-alias! Great! Maybe you and @dheles can team up on a pull request for this issue. 😄

[pdurbin]

Pull request #5487 makes sense to me so I'm moving it to QA. @dheles I'm wondering if a doc update should be included though. Right now I sometimes rely on jumping on a test server and looking up the password for the database in domain.xml. What would I do in the future? The password is in some other file?

By the way, I thought I'd try running the docker-aio tests on this branch but I can't. I'm blocked by #5374. We'd like it to be easy for developers to run the test suite.

[poikilotherm]

Just added a review on #5487 and I second the need of docs on aliases.

[pdurbin]

I just added some docs in 820b1ee and I'm moving this to QA. If anyone doesn't like what I wrote, please speak up. Looking at you @dheles @poikilotherm @landreev 😄

[landreev]

Thanks for adding the docs and moving it along @pdurbin.
The doc entry looks good.

[pdurbin]

@landreev sure.

@kcondon in 9a1a10a I just merged the latest from "develop" into the branch.

[dheles]

@pdurbin I think the docs look good. The only change I might suggest would be linking to the glassfish reference manual: https://javaee.github.io/glassfish/doc/4.0/reference-manual.pdf
It is aging, but accurate (at least for the alias commands) and provides more detail on how to run the commands non-interactively. Its what I used to script the aliasing in the PR.

[pdurbin]

@dheles sounds fine. Please feel free to add a commit for this. Thanks!

[dheles]

@pdurbin so link to the reference manual instead of the payara docs, or in addition to?

[pdurbin]

@dheles well, I do like the direct link to a specific HTML page so I guess I'm thinking "in addition to". If we only link to the PDF you have to say which page it's on.