Preload followers data #5835

kunga · 2024-06-21T14:04:50Z

Changelog entry

...

Changelog category

Not for changelog (changelog entry is not required)

Additional information

Resolves #4744

Latencies become a bit better

https://gist.github.com/kunga/12af041b01499117cb40042ecaf7e1c8

https://nda.ya.ru/t/6HpGDKpA76ZF8r

Before:

After:

kunga · 2024-06-21T14:08:31Z

ydb/core/tablet_flat/flat_part_loader.cpp

-            KeysEnv->Save(cookie, std::move(loaded));
+    if (Stage == EStage::PartView || Stage == EStage::Slice || Stage == EStage::PreloadData) {
+        for (auto& loaded : loadedPages) {
+            LoaderEnv->Save(cookie, std::move(loaded));


Сделал чтобы все страницы шли через кэширующий Env, а то предзагрузка индекса который не пинится может бесконечно зациклиться

kunga · 2024-06-21T14:09:16Z

ydb/core/tablet_flat/flat_part_loader.cpp

+    if (BTreeGroupIndexes) {
+        // Note: preload root nodes only because we don't want to have multiple restarts here
+        for (const auto& meta : BTreeGroupIndexes) {
+            if (meta.LevelCount) getPage(meta.PageId);


Вообще можно и страницу с данными тут грузить, если она одна, но не знаю насколько интересно

github-actions · 2024-06-21T14:13:49Z

⚪ 2024-06-21 14:13:48 UTC Pre-commit check for dd30a8c has started.
⚫ 2024-06-21 14:16:02 UTC Check cancelled

kunga · 2024-06-21T14:14:15Z

ydb/core/tx/datashard/datashard.cpp

@@ -361,6 +361,7 @@ void TDataShard::OnActivateExecutor(const TActorContext& ctx) {
        SyncConfig();
        State = TShardState::Readonly;
        FollowerState = { };
+        Executor()->SetPreloadTablesData({Schema::Sys::TableId, Schema::UserTables::TableId, Schema::Snapshots::TableId});


Если будут варианты покрасивше чем HashSet полем в executor - послушаю

Как вариант можно было бы сделать callback'ом у ITablet, типа нужно ли грузить данные таблицы X. Можно было бы также вызывать во время начальной загрузки, т.к. сейчас это похоже используются только в part switch'ах, и OnActivateExecutor в принципе вызывается только когда уже таблетка готова к работе.

Можно было бы также вызывать во время начальной загрузки

А вот тут по комментарию мне показалось, что в момент загрузки у нас может и не быть системных таблиц на фолловере вовсе?

ydb/ydb/core/tx/datashard/datashard__init.cpp

Line 685 in 5aae686

// Check that TxInit from leader has been already replicated to the follower

Я правда не смог понять, что за TxInit тут имели ввиду.

На фолловерах TxInit (транзакция) не вызывается совсем. Этот комментарий относится к тому, что мы здесь проверяем проинициализировался ли шард вообще - конкретно накатил ли он схему. Потому что фолловер ничего проинициализировать не может, под ним асинхронно схема и данные постоянно меняются.

github-actions · 2024-06-21T14:36:56Z

⚪ 2024-06-21 14:36:55 UTC Pre-commit check for 2471020 has started.
⚪ 2024-06-21 14:39:33 UTC Build linux-x86_64-release-asan is running...
🔴 2024-06-21 14:58:19 UTC Build failed. see the build logs.

github-actions · 2024-06-21T14:42:43Z

⚪ 2024-06-21 14:42:43 UTC Pre-commit check for 2471020 has started.
⚪ 2024-06-21 14:45:19 UTC Build linux-x86_64-release-clang14 is running...
⚫ 2024-06-21 15:08:06 UTC Check cancelled

github-actions · 2024-06-21T14:48:56Z

⚪ 2024-06-21 14:48:56 UTC Pre-commit check for 2471020 has started.
⚪ 2024-06-21 14:51:30 UTC Build linux-x86_64-relwithdebinfo is running...
⚫ 2024-06-21 15:08:07 UTC Check cancelled

github-actions · 2024-06-21T15:14:08Z

⚪ 2024-06-21 15:14:07 UTC Pre-commit check for 4502318 has started.
⚪ 2024-06-21 15:16:37 UTC Build linux-x86_64-release-asan is running...
🟢 2024-06-21 15:24:35 UTC Build successful.
⚪ 2024-06-21 15:24:48 UTC Tests are running...
🔴 2024-06-21 17:25:35 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
10957	10370	0	89	328	170

🟢 2024-06-21 17:26:21 UTC ydbd size 5.5 GiB changed* by +62.8 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `59298b8`	merge: `4502318`	diff	diff %
ydbd size	5 855 347 296 Bytes	5 855 411 552 Bytes	+62.8 KiB	+0.001%
ydbd stripped size	1 255 652 680 Bytes	1 255 665 768 Bytes	+12.8 KiB	+0.001%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-06-21T15:18:41Z

⚪ 2024-06-21 15:18:41 UTC Pre-commit check for 4502318 has started.
⚪ 2024-06-21 15:21:17 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-06-21 15:31:25 UTC Build successful.

github-actions · 2024-06-21T15:33:18Z

⚪ 2024-06-21 15:33:18 UTC Pre-commit check for 4502318 has started.
⚪ 2024-06-21 15:35:58 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-06-21 15:44:58 UTC Build successful.
⚪ 2024-06-21 15:45:12 UTC Tests are running...
🔴 2024-06-21 17:51:54 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
39290	34094	0	14	5165	17

🟢 2024-06-21 17:52:36 UTC ydbd size 8.3 GiB changed* by +55.2 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `59298b8`	merge: `4502318`	diff	diff %
ydbd size	8 961 691 128 Bytes	8 961 747 648 Bytes	+55.2 KiB	+0.001%
ydbd stripped size	488 403 752 Bytes	488 406 248 Bytes	+2.4 KiB	+0.001%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-06-22T13:32:06Z

⚪ 2024-06-22 13:32:05 UTC Pre-commit check for a8854c7 has started.
⚪ 2024-06-22 13:34:51 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-06-22 14:11:55 UTC Build successful.

github-actions · 2024-06-22T13:32:23Z

⚪ 2024-06-22 13:32:23 UTC Pre-commit check for a8854c7 has started.
⚪ 2024-06-22 13:35:07 UTC Build linux-x86_64-release-asan is running...
🟢 2024-06-22 13:55:43 UTC Build successful.
⚪ 2024-06-22 13:56:07 UTC Tests are running...
🔴 2024-06-22 16:04:06 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
10964	10383	0	89	322	170

🟢 2024-06-22 16:04:56 UTC ydbd size 5.5 GiB changed* by +25.2 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `8cc0e51`	merge: `a8854c7`	diff	diff %
ydbd size	5 853 374 536 Bytes	5 853 400 328 Bytes	+25.2 KiB	+0.000%
ydbd stripped size	1 255 453 064 Bytes	1 255 458 472 Bytes	+5.3 KiB	+0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-06-22T13:34:27Z

⚪ 2024-06-22 13:34:27 UTC Pre-commit check for a8854c7 has started.
⚪ 2024-06-22 13:37:10 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-06-22 13:57:00 UTC Build successful.
⚪ 2024-06-22 13:57:20 UTC Tests are running...
🔴 2024-06-22 16:03:44 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
39398	34151	0	8	5222	17

🟢 2024-06-22 16:04:30 UTC ydbd size 8.3 GiB changed* by +39.0 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `8cc0e51`	merge: `a8854c7`	diff	diff %
ydbd size	8 958 642 328 Bytes	8 958 682 256 Bytes	+39.0 KiB	+0.000%
ydbd stripped size	488 345 896 Bytes	488 350 376 Bytes	+4.4 KiB	+0.001%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-06-22T17:54:01Z

⚪ 2024-06-22 17:54:01 UTC Pre-commit check for 53b5f9a has started.
⚪ 2024-06-22 17:56:42 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-06-22 17:59:19 UTC Build successful.
⚪ 2024-06-22 17:59:36 UTC Tests are running...
🔴 2024-06-22 20:06:29 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
39396	34158	0	8	5219	11

🟢 2024-06-22 20:07:17 UTC ydbd size 8.3 GiB changed* by +39.0 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `8cc0e51`	merge: `53b5f9a`	diff	diff %
ydbd size	8 958 642 328 Bytes	8 958 682 256 Bytes	+39.0 KiB	+0.000%
ydbd stripped size	488 345 896 Bytes	488 350 376 Bytes	+4.4 KiB	+0.001%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-06-22T17:54:13Z

⚪ 2024-06-22 17:54:13 UTC Pre-commit check for 53b5f9a has started.
⚪ 2024-06-22 17:56:54 UTC Build linux-x86_64-release-asan is running...
🟢 2024-06-22 17:59:36 UTC Build successful.
⚪ 2024-06-22 17:59:54 UTC Tests are running...
🔴 2024-06-22 20:09:14 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
10968	10374	0	96	328	170

🟢 2024-06-22 20:10:14 UTC ydbd size 5.5 GiB changed* by +25.2 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `8cc0e51`	merge: `53b5f9a`	diff	diff %
ydbd size	5 853 374 536 Bytes	5 853 400 328 Bytes	+25.2 KiB	+0.000%
ydbd stripped size	1 255 453 064 Bytes	1 255 458 472 Bytes	+5.3 KiB	+0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-06-22T17:54:27Z

⚪ 2024-06-22 17:54:27 UTC Pre-commit check for 53b5f9a has started.
⚪ 2024-06-22 17:57:08 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-06-22 17:59:26 UTC Build successful.

github-actions bot added the not-for-changelog label Jun 21, 2024

kunga commented Jun 21, 2024

View reviewed changes

kunga added 11 commits June 21, 2024 15:06

log requests

8bae885

add tests

3a9a626

dont pass pages

f964d9a

preload index

a27cac3

fix endl

15d334a

Preload data

3893fdf

fix checks

5b54f94

pass preload tables

be9a4cb

fix build & flat index

8207c7b

clean up

f9d1df6

fix after rebase

a9c1cd8

kunga force-pushed the preload-followers-data branch from d9887f7 to a9c1cd8 Compare June 21, 2024 15:07

fix tests

173a6ff

fix asan

d7972ca

kunga requested a review from snaury June 24, 2024 06:59

snaury approved these changes Jun 24, 2024

View reviewed changes

kunga merged commit ada3760 into ydb-platform:main Jul 1, 2024
9 of 13 checks passed

This was referenced Jul 2, 2024

YYQ-3398 fixed result set size counting #6157

Merged

YQ-3184 fixed CTAS for s3 sources #6159

Merged

Kqprun fixed fault and logs printing for multi node usage #6295

Merged

This was referenced Jul 9, 2024

Kqprun added request time info printing #6479

Merged

YQ-3405 fixed endless retries for external error #6626

Merged

YQ-3345 fixed WM counters and unit test #6643

Merged

GrigoriyPA mentioned this pull request Jul 24, 2024

YQ WM fix for sls feature flag #7057

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Preload followers data #5835

Preload followers data #5835

kunga commented Jun 21, 2024 •

edited

Loading

kunga Jun 21, 2024

kunga Jun 21, 2024

github-actions bot commented Jun 21, 2024 •

edited

Loading

kunga Jun 21, 2024 •

edited

Loading

snaury Jun 24, 2024

kunga Jun 24, 2024

snaury Jun 26, 2024

github-actions bot commented Jun 21, 2024 •

edited

Loading

github-actions bot commented Jun 21, 2024 •

edited

Loading

github-actions bot commented Jun 21, 2024 •

edited

Loading

github-actions bot commented Jun 21, 2024 •

edited

Loading

github-actions bot commented Jun 21, 2024 •

edited

Loading

github-actions bot commented Jun 21, 2024 •

edited

Loading

github-actions bot commented Jun 22, 2024 •

edited

Loading

github-actions bot commented Jun 22, 2024 •

edited

Loading

github-actions bot commented Jun 22, 2024 •

edited

Loading

github-actions bot commented Jun 22, 2024 •

edited

Loading

github-actions bot commented Jun 22, 2024 •

edited

Loading

github-actions bot commented Jun 22, 2024 •

edited

Loading

Preload followers data #5835

Preload followers data #5835

Conversation

kunga commented Jun 21, 2024 • edited Loading

Changelog entry

Changelog category

Additional information

kunga Jun 21, 2024

Choose a reason for hiding this comment

kunga Jun 21, 2024

Choose a reason for hiding this comment

github-actions bot commented Jun 21, 2024 • edited Loading

kunga Jun 21, 2024 • edited Loading

Choose a reason for hiding this comment

snaury Jun 24, 2024

Choose a reason for hiding this comment

kunga Jun 24, 2024

Choose a reason for hiding this comment

snaury Jun 26, 2024

Choose a reason for hiding this comment

github-actions bot commented Jun 21, 2024 • edited Loading

github-actions bot commented Jun 21, 2024 • edited Loading

github-actions bot commented Jun 21, 2024 • edited Loading

github-actions bot commented Jun 21, 2024 • edited Loading

github-actions bot commented Jun 21, 2024 • edited Loading

github-actions bot commented Jun 21, 2024 • edited Loading

github-actions bot commented Jun 22, 2024 • edited Loading

github-actions bot commented Jun 22, 2024 • edited Loading

github-actions bot commented Jun 22, 2024 • edited Loading

github-actions bot commented Jun 22, 2024 • edited Loading

github-actions bot commented Jun 22, 2024 • edited Loading

github-actions bot commented Jun 22, 2024 • edited Loading

kunga commented Jun 21, 2024 •

edited

Loading

github-actions bot commented Jun 21, 2024 •

edited

Loading

kunga Jun 21, 2024 •

edited

Loading

github-actions bot commented Jun 21, 2024 •

edited

Loading

github-actions bot commented Jun 21, 2024 •

edited

Loading

github-actions bot commented Jun 21, 2024 •

edited

Loading

github-actions bot commented Jun 21, 2024 •

edited

Loading

github-actions bot commented Jun 21, 2024 •

edited

Loading

github-actions bot commented Jun 21, 2024 •

edited

Loading

github-actions bot commented Jun 22, 2024 •

edited

Loading

github-actions bot commented Jun 22, 2024 •

edited

Loading

github-actions bot commented Jun 22, 2024 •

edited

Loading

github-actions bot commented Jun 22, 2024 •

edited

Loading

github-actions bot commented Jun 22, 2024 •

edited

Loading

github-actions bot commented Jun 22, 2024 •

edited

Loading