fix huge blobs volume for start mode withno barrier moving #6366

ivanmorozov333 · 2024-07-06T07:27:24Z

Performance improvement

github-actions · 2024-07-06T07:30:44Z

⚪ 2024-07-06 07:30:44 UTC Pre-commit check for c464e2d has started.
⚪ 2024-07-06 07:33:14 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-07-06 07:38:56 UTC Build successful.

github-actions · 2024-07-06T07:31:11Z

⚪ 2024-07-06 07:31:11 UTC Pre-commit check for c464e2d has started.
⚪ 2024-07-06 07:33:37 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-07-06 08:08:45 UTC Build successful.
⚪ 2024-07-06 08:08:58 UTC Tests are running...
🔴 2024-07-06 08:55:43 UTC Test run completed, no test results found for commit fd60ff0. Please check test log.
🟢 2024-07-06 08:56:27 UTC ydbd size 8.1 GiB changed* by +6.5 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `1fa2be9`	merge: `c464e2d`	diff	diff %
ydbd size	8 721 818 088 Bytes	8 721 824 776 Bytes	+6.5 KiB	+0.000%
ydbd stripped size	476 604 728 Bytes	476 605 560 Bytes	+832 Bytes	+0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}
⚫ 2024-07-06 08:56:28 UTC Check cancelled

github-actions · 2024-07-06T07:31:23Z

⚪ 2024-07-06 07:31:23 UTC Pre-commit check for c464e2d has started.
⚪ 2024-07-06 07:33:51 UTC Build linux-x86_64-release-asan is running...
🟢 2024-07-06 07:56:50 UTC Build successful.
⚪ 2024-07-06 07:57:04 UTC Tests are running...
🔴 2024-07-06 08:55:44 UTC Test run completed, no test results found for commit fd60ff0. Please check test log.
🟢 2024-07-06 08:56:26 UTC ydbd size 5.2 GiB changed* by +15.4 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `1fa2be9`	merge: `c464e2d`	diff	diff %
ydbd size	5 614 212 624 Bytes	5 614 228 416 Bytes	+15.4 KiB	+0.000%
ydbd stripped size	1 209 994 424 Bytes	1 210 001 336 Bytes	+6.8 KiB	+0.001%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}
⚫ 2024-07-06 08:56:27 UTC Check cancelled

github-actions · 2024-07-06T08:59:43Z

⚪ 2024-07-06 08:59:43 UTC Pre-commit check for cc120c4 has started.
⚪ 2024-07-06 09:02:14 UTC Build linux-x86_64-release-clang14 is running...
🔴 2024-07-06 09:04:48 UTC Build failed. see the build logs.

github-actions · 2024-07-06T08:59:47Z

⚪ 2024-07-06 08:59:47 UTC Pre-commit check for cc120c4 has started.
⚪ 2024-07-06 09:02:18 UTC Build linux-x86_64-relwithdebinfo is running...
🔴 2024-07-06 09:04:51 UTC Build failed. see the build logs.
🔴 2024-07-06 09:04:52 UTC Tests run skipped.

github-actions · 2024-07-06T09:00:08Z

⚪ 2024-07-06 09:00:07 UTC Pre-commit check for cc120c4 has started.
⚪ 2024-07-06 09:02:43 UTC Build linux-x86_64-release-asan is running...
🔴 2024-07-06 09:05:04 UTC Build failed. see the build logs.
🔴 2024-07-06 09:05:05 UTC Tests run skipped.

github-actions · 2024-07-06T09:56:42Z

⚪ 2024-07-06 09:56:42 UTC Pre-commit check for f22d77d has started.
⚪ 2024-07-06 09:59:54 UTC Build linux-x86_64-release-asan is running...
🟢 2024-07-06 10:28:18 UTC Build successful.
⚪ 2024-07-06 10:28:31 UTC Tests are running...
🔴 2024-07-06 10:35:18 UTC Test run completed, no test results found for commit 67e208d. Please check test log.
🟢 2024-07-06 10:35:48 UTC ydbd size 5.2 GiB changed* by -195.0 KiB, which is <= 0 Bytes vs main: OK

ydbd size dash	main: `1fa2be9`	merge: `f22d77d`	diff	diff %
ydbd size	5 614 212 624 Bytes	5 614 012 904 Bytes	-195.0 KiB	-0.004%
ydbd stripped size	1 209 994 424 Bytes	1 209 954 040 Bytes	-39.4 KiB	-0.003%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}
⚫ 2024-07-06 10:35:49 UTC Check cancelled

github-actions · 2024-07-06T09:58:09Z

⚪ 2024-07-06 09:58:08 UTC Pre-commit check for f22d77d has started.
⚪ 2024-07-06 10:00:42 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-07-06 10:13:10 UTC Build successful.

github-actions · 2024-07-06T09:58:16Z

⚪ 2024-07-06 09:58:15 UTC Pre-commit check for f22d77d has started.
⚪ 2024-07-06 10:00:50 UTC Build linux-x86_64-relwithdebinfo is running...
⚫ 2024-07-06 10:36:00 UTC Check cancelled

github-actions · 2024-07-06T10:37:27Z

⚪ 2024-07-06 10:37:27 UTC Pre-commit check for 97004fd has started.
⚪ 2024-07-06 10:40:05 UTC Build linux-x86_64-relwithdebinfo is running...
⚫ 2024-07-06 11:06:36 UTC Check cancelled

github-actions · 2024-07-06T10:39:14Z

⚪ 2024-07-06 10:39:14 UTC Pre-commit check for 97004fd has started.
⚪ 2024-07-06 10:41:46 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-07-06 10:53:56 UTC Build successful.

github-actions · 2024-07-06T10:40:01Z

⚪ 2024-07-06 10:40:00 UTC Pre-commit check for 97004fd has started.
⚪ 2024-07-06 10:42:32 UTC Build linux-x86_64-release-asan is running...
⚫ 2024-07-06 11:06:42 UTC Check cancelled

github-actions · 2024-07-06T11:08:39Z

⚪ 2024-07-06 11:08:39 UTC Pre-commit check for 36d2a05 has started.
⚪ 2024-07-06 11:11:50 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-07-06 11:50:49 UTC Build successful.
⚪ 2024-07-06 11:51:08 UTC Tests are running...
🔴 2024-07-06 13:53:07 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
38416	33178	0	1	5221	16

🟢 2024-07-06 13:53:51 UTC ydbd size 8.1 GiB changed* by -298.1 KiB, which is <= 0 Bytes vs main: OK

ydbd size dash	main: `1fa2be9`	merge: `36d2a05`	diff	diff %
ydbd size	8 721 818 088 Bytes	8 721 512 864 Bytes	-298.1 KiB	-0.003%
ydbd stripped size	476 604 728 Bytes	476 587 512 Bytes	-16.8 KiB	-0.004%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-07-06T11:10:33Z

⚪ 2024-07-06 11:10:33 UTC Pre-commit check for 36d2a05 has started.
⚪ 2024-07-06 11:13:11 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-07-06 11:18:36 UTC Build successful.

github-actions · 2024-07-06T11:10:33Z

⚪ 2024-07-06 11:10:33 UTC Pre-commit check for 36d2a05 has started.
⚪ 2024-07-06 11:13:06 UTC Build linux-x86_64-release-asan is running...
🟢 2024-07-06 11:44:31 UTC Build successful.
⚪ 2024-07-06 11:44:53 UTC Tests are running...
🔴 2024-07-06 13:42:16 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
9911	9330	0	85	324	172

🟢 2024-07-06 13:43:04 UTC ydbd size 5.2 GiB changed* by -223.8 KiB, which is <= 0 Bytes vs main: OK

ydbd size dash	main: `1fa2be9`	merge: `36d2a05`	diff	diff %
ydbd size	5 614 212 624 Bytes	5 613 983 464 Bytes	-223.8 KiB	-0.004%
ydbd stripped size	1 209 994 424 Bytes	1 209 946 936 Bytes	-46.4 KiB	-0.004%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-07-07T06:41:59Z

⚪ 2024-07-07 06:41:59 UTC Pre-commit check for 08e6b7e has started.
⚪ 2024-07-07 06:44:31 UTC Build linux-x86_64-release-asan is running...
🟢 2024-07-07 07:12:18 UTC Build successful.
⚪ 2024-07-07 07:12:34 UTC Tests are running...
🔴 2024-07-07 09:10:46 UTC Some tests failed, follow the links below.

Test history | Test log

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
9911	9335	0	82	324	170

🟢 2024-07-07 09:11:35 UTC ydbd size 5.2 GiB changed* by -227.8 KiB, which is <= 0 Bytes vs main: OK

ydbd size dash	main: `0eb74d1`	merge: `08e6b7e`	diff	diff %
ydbd size	5 614 292 712 Bytes	5 614 059 472 Bytes	-227.8 KiB	-0.004%
ydbd stripped size	1 210 025 432 Bytes	1 209 973 848 Bytes	-50.4 KiB	-0.004%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-07-07T06:42:00Z

⚪ 2024-07-07 06:41:59 UTC Pre-commit check for 08e6b7e has started.
⚪ 2024-07-07 06:44:38 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-07-07 06:56:24 UTC Build successful.

github-actions · 2024-07-07T06:42:30Z

⚪ 2024-07-07 06:42:29 UTC Pre-commit check for 08e6b7e has started.
⚪ 2024-07-07 06:45:07 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-07-07 07:28:14 UTC Build successful.
⚪ 2024-07-07 07:28:35 UTC Tests are running...
🟢 2024-07-07 09:15:56 UTC Tests successful.

Test history | Test log

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
38417	33183	0	0	5221	13

🟢 2024-07-07 09:16:47 UTC ydbd size 8.1 GiB changed* by -298.0 KiB, which is <= 0 Bytes vs main: OK

ydbd size dash	main: `0eb74d1`	merge: `08e6b7e`	diff	diff %
ydbd size	8 721 975 384 Bytes	8 721 670 232 Bytes	-298.0 KiB	-0.003%
ydbd stripped size	476 636 792 Bytes	476 619 640 Bytes	-16.8 KiB	-0.004%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

ydb/core/tx/columnshard/blobs_action/abstract/blob_set.h

…orm#6366)

github-actions bot added the performance label Jul 6, 2024

ivanmorozov333 added 6 commits July 7, 2024 09:35

fix huge blobs volume for start mode withno barrier moving

b0ae5fc

correct counters

e17ad9c

fix build

22a53bf

fixes

b233e19

fix

21e3532

rebase fix

71ceb8c

ivanmorozov333 force-pushed the clean_huge_blobs_volume_withno_barrier_move branch from 4d17650 to 71ceb8c Compare July 7, 2024 06:38

zverevgeny reviewed Jul 7, 2024

View reviewed changes

zverevgeny self-requested a review July 7, 2024 08:35

zverevgeny approved these changes Jul 7, 2024

View reviewed changes

zverevgeny self-requested a review July 7, 2024 08:39

ivanmorozov333 merged commit bc77d99 into ydb-platform:main Jul 7, 2024
10 of 12 checks passed

ivanmorozov333 deleted the clean_huge_blobs_volume_withno_barrier_move branch July 7, 2024 10:13

This was referenced Jul 8, 2024

YQ-3367 frozen fq yt-kqp test list #6397

Merged

Kqprun added request time info printing #6479

Merged

YQ-3405 fixed endless retries for external error #6626

Merged

YQ-3345 fixed WM counters and unit test #6643

Merged

zverevgeny pushed a commit to zverevgeny/ydb that referenced this pull request Jul 23, 2024

fix huge blobs volume for start mode withno barrier moving (ydb-platf…

cf97903

…orm#6366)

GrigoriyPA mentioned this pull request Jul 24, 2024

YQ WM fix for sls feature flag #7057

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix huge blobs volume for start mode withno barrier moving #6366

fix huge blobs volume for start mode withno barrier moving #6366

ivanmorozov333 commented Jul 6, 2024

github-actions bot commented Jul 6, 2024 •

edited

Loading

github-actions bot commented Jul 6, 2024 •

edited

Loading

github-actions bot commented Jul 6, 2024 •

edited

Loading

github-actions bot commented Jul 6, 2024 •

edited

Loading

github-actions bot commented Jul 6, 2024 •

edited

Loading

github-actions bot commented Jul 6, 2024 •

edited

Loading

github-actions bot commented Jul 6, 2024 •

edited

Loading

github-actions bot commented Jul 6, 2024 •

edited

Loading

github-actions bot commented Jul 6, 2024 •

edited

Loading

github-actions bot commented Jul 6, 2024 •

edited

Loading

github-actions bot commented Jul 6, 2024 •

edited

Loading

github-actions bot commented Jul 6, 2024 •

edited

Loading

github-actions bot commented Jul 6, 2024 •

edited

Loading

github-actions bot commented Jul 6, 2024 •

edited

Loading

github-actions bot commented Jul 6, 2024 •

edited

Loading

github-actions bot commented Jul 7, 2024 •

edited

Loading

github-actions bot commented Jul 7, 2024 •

edited

Loading

github-actions bot commented Jul 7, 2024 •

edited

Loading

fix huge blobs volume for start mode withno barrier moving #6366

fix huge blobs volume for start mode withno barrier moving #6366

Conversation

ivanmorozov333 commented Jul 6, 2024

github-actions bot commented Jul 6, 2024 • edited Loading

github-actions bot commented Jul 6, 2024 • edited Loading

github-actions bot commented Jul 6, 2024 • edited Loading

github-actions bot commented Jul 6, 2024 • edited Loading

github-actions bot commented Jul 6, 2024 • edited Loading

github-actions bot commented Jul 6, 2024 • edited Loading

github-actions bot commented Jul 6, 2024 • edited Loading

github-actions bot commented Jul 6, 2024 • edited Loading

github-actions bot commented Jul 6, 2024 • edited Loading

github-actions bot commented Jul 6, 2024 • edited Loading

github-actions bot commented Jul 6, 2024 • edited Loading

github-actions bot commented Jul 6, 2024 • edited Loading

github-actions bot commented Jul 6, 2024 • edited Loading

github-actions bot commented Jul 6, 2024 • edited Loading

github-actions bot commented Jul 6, 2024 • edited Loading

github-actions bot commented Jul 7, 2024 • edited Loading

github-actions bot commented Jul 7, 2024 • edited Loading

github-actions bot commented Jul 7, 2024 • edited Loading

github-actions bot commented Jul 6, 2024 •

edited

Loading

github-actions bot commented Jul 6, 2024 •

edited

Loading

github-actions bot commented Jul 6, 2024 •

edited

Loading

github-actions bot commented Jul 6, 2024 •

edited

Loading

github-actions bot commented Jul 6, 2024 •

edited

Loading

github-actions bot commented Jul 6, 2024 •

edited

Loading

github-actions bot commented Jul 6, 2024 •

edited

Loading

github-actions bot commented Jul 6, 2024 •

edited

Loading

github-actions bot commented Jul 6, 2024 •

edited

Loading

github-actions bot commented Jul 6, 2024 •

edited

Loading

github-actions bot commented Jul 6, 2024 •

edited

Loading

github-actions bot commented Jul 6, 2024 •

edited

Loading

github-actions bot commented Jul 6, 2024 •

edited

Loading

github-actions bot commented Jul 6, 2024 •

edited

Loading

github-actions bot commented Jul 6, 2024 •

edited

Loading

github-actions bot commented Jul 7, 2024 •

edited

Loading

github-actions bot commented Jul 7, 2024 •

edited

Loading

github-actions bot commented Jul 7, 2024 •

edited

Loading