adam-submit is not exiting until I hit ctrl+C

[bioinfornatics]

Dear,

Firstly, thanks for this useful framework.
I encounter an unwanted behavior while using adam-submit, indeed when I use a command like:

bin/adam-submit   --conf spark.task.maxFailures=20 \
                  --master yarn  \
                  --deploy-mode cluster  \
                  --driver-memory 4g \
                  --driver-cores 2 \
                  --executor-memory 10g \
                  --executor-cores 10 \
                  --num-executors 3  \
                  --verbose \
                  --name chr1 \
                  -- transformGenotypes   /user/jmercier/data/1000G/legacy/ALL.chr1.phase3_shapeit2_mvncall_integrated_v5a.20130502.genotypes.vcf.gz \
                                          /user/jmercier/ALL.chr1.genotypes.adam \
                  > ALL.chr1.genotypes.log

The application never end, I need to hit ctrl+C to stop it.

so I would like to know if you have the same behavior?

If yes, maybe you should call stop() on context to exit your program cleanly at the end of run() function at line 126

Best regards

[bioinfornatics]

up

[heuermh]

Sorry for the long delay on this issue, I am not able to reproduce.

Note running on Yarn with --deploy-mode cluster, the logs output by adam-submit are going to be fairly worthless. Better to use the Spark UI or other methods to view the logs on the Spark executors.