v1.9.0-rc.0: long incoming req/resp handler in network thread

[twoeths]

Describe the bug

Long incoming request handler for beacon_blocks_by_root, ping and status

Expected behavior

same incoming request handler like in stable

Steps to reproduce

No response

Additional context

No response

Operating system

Linux

Lodestar version or commit hash

v1.9.0-rc.0

[twoeths]

I added below log to this line

lodestar/packages/reqresp/src/response/index.ts

Line 60 in 8490369

logger.debug("Req  received before decoding request", logCtx);

it logged a delay in requestDecode() function or I/O lag

it took 1s to receive and decode a ping request

May-30 09:39:14.734[network]         debug: Req  received before decoding request method=ping, client=Lighthouse, peer=16...8DShj2, requestId=5273
May-30 09:39:15.670[network]         debug: Req  received method=ping, client=Lighthouse, peer=16...8DShj2, requestId=5273

it took 3s to receive and decode a beacon_blocks_by_range request

May-30 09:44:39.768[network]         debug: Req  received before decoding request method=beacon_blocks_by_range, client=Lighthouse, peer=16...kY2mne, requestId=7098
May-30 09:44:43.069[network]         debug: Req  received method=beacon_blocks_by_range, client=Lighthouse, peer=16...kY2mne, requestId=7098

[twoeths]

@dapplion found I/O on the network thread to be super lag

[twoeths]

issue was resolved, this is on a mainnet node with useWorker = true

it's even better than unstable mainnet node where useWorker=false