Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

adds keep-alive-interval to repair QUIC transport config #33866

Merged
merged 1 commit into from
Nov 8, 2023
Merged

adds keep-alive-interval to repair QUIC transport config #33866

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 13 additions & 3 deletions core/src/repair/quic_endpoint.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ use {
log::error,
quinn::{
ClientConfig, ConnectError, Connecting, Connection, ConnectionError, Endpoint,
EndpointConfig, ReadError, ReadToEndError, RecvStream, SendStream, ServerConfig,
TokioRuntime, TransportConfig, VarInt, WriteError,
EndpointConfig, IdleTimeout, ReadError, ReadToEndError, RecvStream, SendStream,
ServerConfig, TokioRuntime, TransportConfig, VarInt, WriteError,
},
rcgen::RcgenError,
rustls::{Certificate, PrivateKey},
Expand Down Expand Up @@ -46,7 +46,13 @@ const CONNECT_SERVER_NAME: &str = "solana-repair";
const CLIENT_CHANNEL_BUFFER: usize = 1 << 14;
const ROUTER_CHANNEL_BUFFER: usize = 64;
const CONNECTION_CACHE_CAPACITY: usize = 3072;

// Transport config.
// Repair randomly samples peers, uses bi-directional streams and generally has
// low to moderate load and so is configured separately from other protocols.
const KEEP_ALIVE_INTERVAL: Duration = Duration::from_secs(4);
const MAX_CONCURRENT_BIDI_STREAMS: VarInt = VarInt::from_u32(512);
const MAX_IDLE_TIMEOUT: Duration = Duration::from_secs(10);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see definitions for these in sdk/src/quic.rs. Can you add comments about why new definitions are needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ones in sdk are used for tpu, which has very different properties than repair. In particular tpu packets are only forwarded to the upcoming leaders which we know in advance who they are, it is a uni-directional traffic, and has a very different load and latency tolerance. Repair is bi-directional request-response, peers are randomly sampled, etc.

I pretty much rather not to reuse the same parameters at this stage until this code is stable and repair over quic protocol is fully rolled out over mainnet. Until then reusing same parameters can introduce regressions on one protocol while trying to optimize another protocol.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think just add some code comments on the reasoning will help.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.


const CONNECTION_CLOSE_ERROR_CODE_SHUTDOWN: VarInt = VarInt::from_u32(1);
const CONNECTION_CLOSE_ERROR_CODE_DROPPED: VarInt = VarInt::from_u32(2);
Expand Down Expand Up @@ -195,11 +201,15 @@ fn new_client_config(cert: Certificate, key: PrivateKey) -> Result<ClientConfig,
}

fn new_transport_config() -> TransportConfig {
let max_idle_timeout = IdleTimeout::try_from(MAX_IDLE_TIMEOUT).unwrap();
let mut config = TransportConfig::default();
// Disable datagrams and uni streams.
config
.datagram_receive_buffer_size(None)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment this to disable datagrams?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

.keep_alive_interval(Some(KEEP_ALIVE_INTERVAL))
.max_concurrent_bidi_streams(MAX_CONCURRENT_BIDI_STREAMS)
.max_concurrent_uni_streams(VarInt::from(0u8))
.datagram_receive_buffer_size(None);
.max_idle_timeout(Some(max_idle_timeout));
config
}

Expand Down