Skip to content

Commit

Permalink
fix spark-6033, clarify the spark.worker.cleanup behavior in standalo…
Browse files Browse the repository at this point in the history
…ne mode

jira case spark-6033 https://issues.apache.org/jira/browse/SPARK-6033

In standalone deploy mode, the cleanup will only remove the stopped application's directories.

The original description about the cleanup behavior is incorrect.

Author: 许鹏 <peng.xu@fraudmetrix.cn>

Closes #4803 from hseagle/spark-6033 and squashes the following commits:

927a6a0 [许鹏] fix the incorrect description about the spark.worker.cleanup in standalone mode
  • Loading branch information
许鹏 authored and Andrew Or committed Feb 27, 2015
1 parent 14e042b commit e751f8f
Showing 1 changed file with 1 addition and 2 deletions.
3 changes: 1 addition & 2 deletions docs/spark-standalone.md
Original file line number Diff line number Diff line change
Expand Up @@ -209,8 +209,7 @@ SPARK_WORKER_OPTS supports the following system properties:
<td>false</td>
<td>
Enable periodic cleanup of worker / application directories. Note that this only affects standalone
mode, as YARN works differently. Applications directories are cleaned up regardless of whether
the application is still running.
mode, as YARN works differently. Only the directories of stopped applications are cleaned up.
</td>
</tr>
<tr>
Expand Down

0 comments on commit e751f8f

Please sign in to comment.