Skip to content

Commit

Permalink
[SPARK-33840][DOCS] Add spark.sql.files.minPartitionNum to performenc…
Browse files Browse the repository at this point in the history
…e tuning doc

### What changes were proposed in this pull request?

Add `spark.sql.files.minPartitionNum` and it's description to sql-performence-tuning.md.

### Why are the changes needed?

Help user to find it.

### Does this PR introduce _any_ user-facing change?

Yes, it's the doc.

### How was this patch tested?

Pass CI.

Closes #30838 from ulysses-you/SPARK-33840.

Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: HyukjinKwon <gurwls223@apache.org>
(cherry picked from commit bc46d27)
Signed-off-by: HyukjinKwon <gurwls223@apache.org>
  • Loading branch information
ulysses-you authored and HyukjinKwon committed Dec 18, 2020
1 parent 8a269c7 commit d89c87e
Showing 1 changed file with 10 additions and 0 deletions.
10 changes: 10 additions & 0 deletions docs/sql-performance-tuning.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,16 @@ that these options will be deprecated in future release as more optimizations ar
</td>
<td>2.0.0</td>
</tr>
<tr>
<td><code>spark.sql.files.minPartitionNum</code></td>
<td>Default Parallelism</td>
<td>
The suggested (not guaranteed) minimum number of split file partitions. If not set, the default
value is `spark.default.parallelism`. This configuration is effective only when using file-based
sources such as Parquet, JSON and ORC.
</td>
<td>3.1.0</td>
</tr>
<tr>
<td><code>spark.sql.broadcastTimeout</code></td>
<td>300</td>
Expand Down

0 comments on commit d89c87e

Please sign in to comment.