Skip to content
This repository has been archived by the owner on Jan 13, 2025. It is now read-only.

expands transport configs in turbine QUIC endpoint #33864

Merged
merged 1 commit into from
Nov 8, 2023
Merged

expands transport configs in turbine QUIC endpoint #33864

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 18 additions & 3 deletions turbine/src/quic_endpoint.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@ use {
log::error,
quinn::{
ClientConfig, ConnectError, Connecting, Connection, ConnectionError, Endpoint,
EndpointConfig, SendDatagramError, ServerConfig, TokioRuntime, TransportConfig, VarInt,
EndpointConfig, IdleTimeout, SendDatagramError, ServerConfig, TokioRuntime,
TransportConfig, VarInt,
},
rcgen::RcgenError,
rustls::{Certificate, PrivateKey},
Expand Down Expand Up @@ -39,10 +40,17 @@ use {
const CLIENT_CHANNEL_BUFFER: usize = 1 << 14;
const ROUTER_CHANNEL_BUFFER: usize = 64;
const CONNECTION_CACHE_CAPACITY: usize = 3072;
const INITIAL_MAXIMUM_TRANSMISSION_UNIT: u16 = 1280;
const ALPN_TURBINE_PROTOCOL_ID: &[u8] = b"solana-turbine";
const CONNECT_SERVER_NAME: &str = "solana-turbine";

// Transport config.
const DATAGRAM_RECEIVE_BUFFER_SIZE: usize = 256 * 1024 * 1024;
const DATAGRAM_SEND_BUFFER_SIZE: usize = 128 * 1024 * 1024;
const INITIAL_MAXIMUM_TRANSMISSION_UNIT: u16 = MINIMUM_MAXIMUM_TRANSMISSION_UNIT;
const KEEP_ALIVE_INTERVAL: Duration = Duration::from_secs(4);
const MAX_IDLE_TIMEOUT: Duration = Duration::from_secs(10);
const MINIMUM_MAXIMUM_TRANSMISSION_UNIT: u16 = 1280;
Comment on lines +46 to +52
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Were these values found/picked empirically? The MTU and durations are seemingly simple; however, the buffer size ones seem a little less straight forward to me in how the numbers were picked

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the buffer sizes I dug the default values in the quinn library and increased them by a couple of orders of magnitude because the peak of the turbine traffic can be pretty high.
https://docs.rs/quinn-proto/0.10.2/src/quinn_proto/config.rs.html#296-330

My primary goal here is to make these core configs explicit. Hopefully, the incremental testing on testnet will allow to further fine tune these if needed.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it. Maybe if/once we determine that QUIC is a suitable replacement for turbine, we can circle back and leave some paper trail comments. Hypothetically, given a block duration and block size (in shreds), we should be able to come up with a ballpark "expected load".

Not saying I think these numbers have to be a "tight fit"; rather, some rough numbers + a comment of leaving sufficient margin would be nice for the next person who is looking through the code


const CONNECTION_CLOSE_ERROR_CODE_SHUTDOWN: VarInt = VarInt::from_u32(1);
const CONNECTION_CLOSE_ERROR_CODE_DROPPED: VarInt = VarInt::from_u32(2);
const CONNECTION_CLOSE_ERROR_CODE_INVALID_IDENTITY: VarInt = VarInt::from_u32(3);
Expand Down Expand Up @@ -173,11 +181,18 @@ fn new_client_config(cert: Certificate, key: PrivateKey) -> Result<ClientConfig,
}

fn new_transport_config() -> TransportConfig {
let max_idle_timeout = IdleTimeout::try_from(MAX_IDLE_TIMEOUT).unwrap();
let mut config = TransportConfig::default();
config
.datagram_receive_buffer_size(Some(DATAGRAM_RECEIVE_BUFFER_SIZE))
.datagram_send_buffer_size(DATAGRAM_SEND_BUFFER_SIZE)
.initial_mtu(INITIAL_MAXIMUM_TRANSMISSION_UNIT)
.keep_alive_interval(Some(KEEP_ALIVE_INTERVAL))
.max_concurrent_bidi_streams(VarInt::from(0u8))
.max_concurrent_uni_streams(VarInt::from(0u8))
.initial_mtu(INITIAL_MAXIMUM_TRANSMISSION_UNIT);
.max_idle_timeout(Some(max_idle_timeout))
.min_mtu(MINIMUM_MAXIMUM_TRANSMISSION_UNIT)
.mtu_discovery_config(None);
config
}

Expand Down