Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fork sync targets are handled inefficiently #525

Open
2 tasks done
altonen opened this issue Nov 18, 2022 · 1 comment
Open
2 tasks done

Fork sync targets are handled inefficiently #525

altonen opened this issue Nov 18, 2022 · 1 comment
Labels
I9-optimisation An enhancement to provide better overall performance in terms of time-to-completion for a task. T10-tests This PR/Issue is related to tests.

Comments

@altonen
Copy link
Contributor

altonen commented Nov 18, 2022

Is there an existing issue?

  • I have searched the existing issues

Experiencing problems? Have you tried our Stack Exchange first?

  • This is not a support question.

Description of bug

While refactoring the syncing code, I looked into sync_to_tip_when_we_sync_with_multiple_peers and while the test itself is dubious (more on that below), it also showed that fork sync targets are stored and handled inefficiently. The test takes over a minute because it spends most of the time looping over fork sync targets and checking if they can be removed or downloaded. Blocks of the same fork are not associated in the fork sync targets queue so if block A is too far in the future, all of its descendants are too so there is no reason to check them separately. The code should be improved to store the targets differently so that we don't have to check every block of the fork in order to determine whether to download it or not.

I'm also not entirely sure if the test is testing what is supposed to. It adds 10000 and 5000 blocks for two different peers and buffers the block announcements so after the blocks have been imported and the network is polled for the first time, the network is flooded with thousands of block announcements. Because peers are idling when the announcements are received, they treat them as fork sync targets and third peer has something like 4000+ fork sync targets even though there are only two forks. And because the fork sync targets are handled inefficiently, the peers spend lots of time going over the list of targets and checking which can be removed and which can be downloaded.

The test probably should be rewritten to not spam the block announcements and instead download the 2 forks gracefully and without unnecessarily checking the same things over and over again.

Steps to reproduce

No response

@altonen altonen added I5-tests I9-optimisation An enhancement to provide better overall performance in terms of time-to-completion for a task. labels Nov 18, 2022
@timorl timorl mentioned this issue Aug 24, 2023
@altonen altonen transferred this issue from paritytech/substrate Aug 24, 2023
@the-right-joyce the-right-joyce added T10-tests This PR/Issue is related to tests. and removed I5-tests labels Aug 25, 2023
@dmitry-markin
Copy link
Contributor

I'm not sure that there are two forks in sync_to_tip_when_we_sync_with_multiple_peers, not one. Aren't we pushing identical blocks to both peers?

bkchr pushed a commit that referenced this issue Apr 10, 2024
* relay Rialto messages to Millau

* impl SubmitRialtoToMillauMessage

* fmt
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
I9-optimisation An enhancement to provide better overall performance in terms of time-to-completion for a task. T10-tests This PR/Issue is related to tests.
Projects
Status: Backlog 🗒
Development

Successfully merging a pull request may close this issue.

3 participants