Security: Spawn each initial peer handshake in a separate task, Credit: Equilibrium

[teor2345]

Is your feature request related to a problem? Please describe.

Zebra can hang if there are a small number of initial peers. (It eventually continues after 3*75 seconds, but that's a long time to wait.)

There might also be some potential deadlocks between the initial peer handshakes, the candidate set, and the peer set.

Reported by Niklas Long of Equilibrium.

Describe the solution you'd like

Spawn each initial peer handshake in a separate task, so they can make progress independently.

Additional context

This is the same fix as #1950, but for the initial peer handshakes.

This is a follow-up to #2154, which fixed part of this security issue.

[teor2345]

This is a high-risk change, so we'll leave it until after NU5 testnet activation.

[teor2345]

Each initial handshake is a future, so they all run in the same task:

zebra/zebra-network/src/peer_set/initialize.rs

Lines 292 to 300 in a6d56b2

	async move {
	// Rate-limit the connection, sleeping for an interval according
	// to its index in the list.
	sleep(constants::MIN_PEER_CONNECTION_INTERVAL.saturating_mul(i as u32)).await;
	outbound_connector
	.oneshot(req)
	.map_err(move |e| (addr, e))
	.await
	}

Instead, we need to spawn a new task to run each future, like this code:

zebra/zebra-network/src/peer_set/initialize.rs

Lines 510 to 527 in a6d56b2

	// Construct a handshake future but do not drive it yet....
	let handshake = handshaker.call(HandshakeRequest {
	tcp_stream,
	connected_addr,
	connection_tracker,
	});
	// ... instead, spawn a new task to handle this connection
	{
	let mut peerset_tx = peerset_tx.clone();
	tokio::spawn(
	async move {
	if let Ok(client) = handshake.await {
	let _ = peerset_tx.send(Ok(Change::Insert(addr, client))).await;
	}
	}
	.instrument(handshaker_span),
	);
	}