Skip to content

Commit

Permalink
[FLINK-29130][state] Correct the doc description of local-recovery
Browse files Browse the repository at this point in the history
  • Loading branch information
Myasuka committed Aug 31, 2022
1 parent 55cf3b3 commit 2fa5749
Show file tree
Hide file tree
Showing 4 changed files with 11 additions and 12 deletions.
4 changes: 2 additions & 2 deletions docs/content.zh/docs/ops/state/large_state_tuning.md
Original file line number Diff line number Diff line change
Expand Up @@ -292,10 +292,10 @@ also cover operator state and timers.*

The following state backends can support task-local recovery.

- FsStateBackend: task-local recovery is supported for keyed state. The implementation will duplicate the state to a local file. This can introduce additional write costs
- **HashMapStateBackend**: task-local recovery is supported for keyed state. The implementation will duplicate the state to a local file. This can introduce additional write costs
and occupy local disk space. In the future, we might also offer an implementation that keeps task-local state in memory.

- RocksDBStateBackend: task-local recovery is supported for keyed state. For *full checkpoints*, state is duplicated to a local file. This can introduce additional write costs
- **EmbeddedRocksDBStateBackend**: task-local recovery is supported for keyed state. For *full checkpoints*, state is duplicated to a local file. This can introduce additional write costs
and occupy local disk space. For *incremental snapshots*, the local state is based on RocksDB's native checkpointing mechanism. This mechanism is also used as the first step
to create the primary copy, which means that in this case no additional cost is introduced for creating the secondary copy. We simply keep the native checkpoint directory around
instead of deleting it after uploading to the distributed store. This local copy can share active files with the working directory of RocksDB (via hard links), so for active
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
<td><h5>state.backend.local-recovery</h5></td>
<td style="word-wrap: break-word;">false</td>
<td>Boolean</td>
<td>This option configures local recovery for this state backend. By default, local recovery is deactivated. Local recovery currently only covers keyed state backends. Currently, the MemoryStateBackend does not support local recovery and ignores this option.</td>
<td>This option configures local recovery for this state backend. By default, local recovery is deactivated. Local recovery currently only covers keyed state backends (including both the EmbeddedRocksDBStateBackend and the HashMapStateBackend).</td>
</tr>
<tr>
<td><h5>state.checkpoint-storage</h5></td>
Expand Down Expand Up @@ -60,7 +60,7 @@
<td><h5>taskmanager.state.local.root-dirs</h5></td>
<td style="word-wrap: break-word;">(none)</td>
<td>String</td>
<td>The config parameter defining the root directories for storing file-based state for local recovery. Local recovery currently only covers keyed state backends. Currently, MemoryStateBackend does not support local recovery and ignore this option</td>
<td>The config parameter defining the root directories for storing file-based state for local recovery. Local recovery currently only covers keyed state backends.</td>
</tr>
</tbody>
</table>
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@
<td><h5>state.backend.local-recovery</h5></td>
<td style="word-wrap: break-word;">false</td>
<td>Boolean</td>
<td>This option configures local recovery for this state backend. By default, local recovery is deactivated. Local recovery currently only covers keyed state backends. Currently, the MemoryStateBackend does not support local recovery and ignores this option.</td>
<td>This option configures local recovery for this state backend. By default, local recovery is deactivated. Local recovery currently only covers keyed state backends (including both the EmbeddedRocksDBStateBackend and the HashMapStateBackend).</td>
</tr>
<tr>
<td><h5>state.checkpoints.num-retained</h5></td>
Expand All @@ -54,7 +54,7 @@
<td><h5>taskmanager.state.local.root-dirs</h5></td>
<td style="word-wrap: break-word;">(none)</td>
<td>String</td>
<td>The config parameter defining the root directories for storing file-based state for local recovery. Local recovery currently only covers keyed state backends. Currently, MemoryStateBackend does not support local recovery and ignore this option</td>
<td>The config parameter defining the root directories for storing file-based state for local recovery. Local recovery currently only covers keyed state backends.</td>
</tr>
</tbody>
</table>
Original file line number Diff line number Diff line change
Expand Up @@ -180,17 +180,17 @@ public class CheckpointingOptions {
* This option configures local recovery for this state backend. By default, local recovery is
* deactivated.
*
* <p>Local recovery currently only covers keyed state backends. Currently, MemoryStateBackend
* and HashMapStateBackend do not support local recovery and ignore this option.
* <p>Local recovery currently only covers keyed state backends (including both the
* EmbeddedRocksDBStateBackend and the HashMapStateBackend).
*/
@Documentation.Section(Documentation.Sections.COMMON_STATE_BACKENDS)
public static final ConfigOption<Boolean> LOCAL_RECOVERY =
ConfigOptions.key("state.backend.local-recovery")
.defaultValue(false)
.withDescription(
"This option configures local recovery for this state backend. By default, local recovery is "
+ "deactivated. Local recovery currently only covers keyed state backends. Currently, the MemoryStateBackend "
+ "does not support local recovery and ignores this option.");
+ "deactivated. Local recovery currently only covers keyed state backends "
+ "(including both the EmbeddedRocksDBStateBackend and the HashMapStateBackend).");

/**
* The config parameter defining the root directories for storing file-based state for local
Expand All @@ -205,8 +205,7 @@ public class CheckpointingOptions {
.noDefaultValue()
.withDescription(
"The config parameter defining the root directories for storing file-based state for local "
+ "recovery. Local recovery currently only covers keyed state backends. Currently, MemoryStateBackend does "
+ "not support local recovery and ignore this option");
+ "recovery. Local recovery currently only covers keyed state backends.");

// ------------------------------------------------------------------------
// Options specific to the file-system-based state backends
Expand Down

0 comments on commit 2fa5749

Please sign in to comment.