Fix disconnecting peers during snap sync

[asdacap]

Fix disconnecting peers during snap sync by incrementally increase snap sync's response byte limit until its latency goes too high. Latency threshold was set low since 3s+ still causes some snap sync failure. Previously, snap request was too large and was also causing FCU peer refresh to fail which trigger peer disconnect.

Alternatively just reducing the respone bytes to fixed 200KB works also.

Changes:

• Auto adjust response bytes.

Types of changes

What types of changes does your code introduce?
Put an x in the boxes that apply

• Bugfix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation Update
• Code style update (formatting, renaming)
• Refactoring (no functional changes, no api changes)
• Build related changes
• Other (please describe):

Testing

Requires testing

• Yes
• No

In case you checked yes, did you write tests??

• Yes

• No

• Goerli can sync to head.

• Performance implication is unclear.

Further comments (optional)

If this is a relatively large or complex change, kick off the discussion by explaining why you chose the solution you did and what alternatives you considered, etc...

[dceleda]

@asdacap do you know why it (2_000_000) wasn't a problem before and suddenly we started to have the problem with peers?

[asdacap]

I've noticed it before, but I've increase FCU PeerRefresher's timeout to compensate. It's unclear if there are any trend in the network that causes it to be a bigger problem.

[MarekM25]

For me it looks good. It would be great to get approval from @dceleda too

[dceleda]

@asdacap have you compared sync times before and after the change?
Any improvement?

[dceleda]

I've run it on my Fast VM and the time is not worse than it used to be - just below 2h.
Looks good then :)

[asdacap]

About 20% improvement on goerli. No significant change on mainnet for some reason.... Positive anecdotes from Kamil.