Rethink state sync

[bowenwang1996]

Currently when we do state sync, we request state parts of size at most 1MB. However, when the state size is on the order of gigabytes, it means that a node needs to request thousands of parts and each part takes some time to compute. Since we request all parts at once, this causes unnecessary increase in network traffic, especially since the requests and responses are also routed. We should maybe reconsider the entire approach and see how we make it better when the state size is very large.

[bowenwang1996]

cc @mikhailOK

[mikhailOK]

Potential ideas:

• Try to speed up computing state parts with a faster trie iterator
• Pre-compute state parts in the background when epoch starts (currently we compute state parts upon receiving a request)
• Manage incoming state part requests. Don't respond twice to duplicate requests, reject instantly if there are too many queued up.

[stale]

This issue has been automatically marked as stale because it has not had recent activity in the last 2 months.
It will be closed in 7 days if no further activity occurs.
Thank you for your contributions.

[stale]

This issue has been automatically marked as stale because it has not had recent activity in the last 2 months.
It will be closed in 7 days if no further activity occurs.
Thank you for your contributions.

[stale]

This issue has been automatically marked as stale because it has not had recent activity in the last 2 months.
It will be closed in 7 days if no further activity occurs.
Thank you for your contributions.

[Longarithm]

Tracked around #8545.