Zero configuration fte #14957
Zero configuration fte #14957
Conversation
|
Resolves #14955 |
| @@ -284,6 +291,11 @@ protected void setup(Binder binder) | |||
| newExporter(binder).export(PauseMeter.class).withGeneratedName(); | |||
|
|
|||
| configBinder(binder).bindConfig(MemoryManagerConfig.class); | |||
| configBinder(binder).bindConfigDefaults(MemoryManagerConfig.class, config -> { | |||
There was a problem hiding this comment.
is that worth mentioning in the comment in config classes that defaults are overriden here?
Or maybe it would be more readable to define fteDefaultOverrides() in config class itself and only bind it here?
Disable redistribute_writes to have the fixes number of writer tasks independent of the execution mode (fte or pipelined)
When FTE is enabled the system can easily retry a single task. It is no longer necessary to be ultra conservative when acknowledging that a certain task is lost.
arhimondr
force-pushed
the
zero-configuration-fte
branch
from
1bf9b19 to
eb973b7
Compare
|
Updated |
| public void applyFaultTolerantExecutionDefaults() | ||
| { | ||
| exchangeCompressionEnabled = true; | ||
| } |
There was a problem hiding this comment.
while it's good to keep FTE defaults in same place as the other configs the method being public can lead to misuse. Config objects from Airlift are meant to be immutable so using the method anywhere except bindConfigDefaults would not have intended effects.
Would declaring a constant here and using it in bindConfigDefaults be better in that regards?
There was a problem hiding this comment.
Does Airlift do anything special with set* methods?
There was a problem hiding this comment.
Does Airlift do anything special with set* methods?
Not sure, didn't check.
You can obviously call set* after the fact but since the config is Guice managed and also affects other modules via conditional modules the point in time at which the set* is called can have different impact and makes reasoning about whole thing problematic.
An example is #14921 (not related to modifying the config object but regarding the effect when the config is modified can have).
Description
Set recommended settings automatically when FTE is enabled.
Non-technical explanation
Currently there are certain settings that we recommend changing when enabling FTE. This is less than ideal as it makes it more difficult to change the values of these setting in the future if necessary as they may be already configured by the user and outside of our control
Release notes
( ) This is not user-visible or docs only and no release notes are required.
( ) Release notes are required, please propose a release note for me.
(X) Release notes are required, with the following suggested text: