Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Preload indexes on followers #4744

Closed
kunga opened this issue May 22, 2024 · 2 comments · Fixed by #5835
Closed

Preload indexes on followers #4744

kunga opened this issue May 22, 2024 · 2 comments · Fixed by #5835
Assignees
@kunga

Comments

@kunga
Copy link
Member

kunga commented May 22, 2024 •
edited
Loading

In 24-1 we removed passing of FlatIndex pages from leaders to followers and it seems to be a cause of high tail latency on followers

To resolve this we should:

  • Reproduce that behaviour on a slice
  • Remove extra disk reads on followers during transaction execution process
    • Fully preload Sys, UserTables, Snapshots tables
    • Preload indexes
  • Remove passing any other pages from leader to followers (should be loaded during a part switch)

Connected to YDBREQUESTS-3667, KIKIMR-18751
@kunga kunga self-assigned this May 22, 2024
@kunga kunga mentioned this issue May 23, 2024
Closed
@kunga
Copy link
Member Author

kunga commented May 24, 2024 •
edited
Loading

Experiment: 1 thread read from a follower with simultaneous writes

Timings:

  • 00-10 - 1 writer, batch size 1
  • 10-20 - 10 writers, batch size 1
  • 20-30 - 50 writers, batch size 1
  • 30-40 - 50 writers, batch size 100

ydb-stable-23-4-11

image

ydb-stable-24-1-14-hotfix-7

image

@kunga kunga changed the title Investigate and fix followers latency Preload indexes on followers May 27, 2024
@kunga kunga mentioned this issue May 27, 2024
Open
62 tasks
@kunga kunga mentioned this issue Jun 21, 2024
Merged
@kunga
Copy link
Member Author

kunga commented Sep 28, 2024

Improved latency in database:
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kunga kunga

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Preload followers data kunga/ydb
1 participant
@kunga