Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix external reference precharging in datashard read iterators #8592

Merged
merged 1 commit into from
Sep 2, 2024

Conversation

snaury
Copy link
Member

@snaury snaury commented Sep 2, 2024

Changelog entry

...

Changelog category

  • Bugfix

Additional information

PR #5758 implemented external reference precharging, which helped to reduce latency bound page faults when reading multiple rows with data stored in external blobs. Unfortunately it turned out to have a bug, where a shard restart could cause ReadActor to retry based on an incorrect state, which would in turn cause missing rows. Since we cannot get the previous iteration key efficiently it's better to never split results across external reference precharging and due a full page fault instead.

Fixes #7769.

@github-actions github-actions bot added the bugfix label Sep 2, 2024
Copy link

github-actions bot commented Sep 2, 2024

2024-09-02 10:06:31 UTC Pre-commit check linux-x86_64-release-clang14 for 59cdebb has started.
2024-09-02 10:09:31 UTC ya make is running...
🟢 2024-09-02 10:17:45 UTC Build successful.

Copy link

github-actions bot commented Sep 2, 2024

2024-09-02 10:14:48 UTC Pre-commit check linux-x86_64-release-asan for 59cdebb has started.
2024-09-02 10:17:30 UTC ya make is running...
🔴 2024-09-02 11:49:06 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
10073 10009 0 18 25 21

🟢 2024-09-02 11:49:58 UTC Build successful.
🟢 2024-09-02 11:50:28 UTC ydbd size 5.5 GiB changed* by -5.3 KiB, which is <= 0 Bytes vs main: OK

ydbd size dash main: 84523e8 merge: 59cdebb diff diff %
ydbd size 5 953 683 440 Bytes 5 953 677 992 Bytes -5.3 KiB -0.000%
ydbd stripped size 1 489 314 256 Bytes 1 489 313 744 Bytes -512 Bytes -0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Sep 2, 2024

2024-09-02 10:16:26 UTC Pre-commit check linux-x86_64-relwithdebinfo for 59cdebb has started.
2024-09-02 10:19:10 UTC ya make is running...
🟡 2024-09-02 11:25:51 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
14483 13089 0 16 1360 18

2024-09-02 11:27:25 UTC ya make is running... (failed tests rerun, try 2)
🟢 2024-09-02 11:35:25 UTC Tests successful.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
33 (only retried tests) 20 0 0 0 13

🟢 2024-09-02 11:35:32 UTC Build successful.
🟡 2024-09-02 11:36:08 UTC ydbd size 8.3 GiB changed* by +715.6 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 1af55df merge: 59cdebb diff diff %
ydbd size 8 915 367 680 Bytes 8 916 100 448 Bytes +715.6 KiB +0.008%
ydbd stripped size 481 569 224 Bytes 481 590 920 Bytes +21.2 KiB +0.005%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@snaury snaury requested a review from SammyVimes September 2, 2024 11:56
@snaury snaury marked this pull request as ready for review September 2, 2024 11:56
Copy link
Collaborator

@SammyVimes SammyVimes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@snaury snaury merged commit 37c91d7 into ydb-platform:main Sep 2, 2024
11 of 13 checks passed
@snaury snaury deleted the bugfix-7769-read-extblobs-restart branch September 2, 2024 13:08
@snaury snaury self-assigned this Sep 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

DataShard: read iterators may have a broken external reference precharging
2 participants