Async fetch portion data access info #11246

ivanmorozov333 · 2024-11-04T13:41:40Z

Improvement
Performance improvement

github-actions · 2024-11-04T13:45:26Z

⚪ 2024-11-04 13:45:25 UTC Pre-commit check linux-x86_64-relwithdebinfo for be78ed0 has started.
⚪ 2024-11-04 13:45:36 UTC Artifacts will be uploaded here
⚪ 2024-11-04 13:48:49 UTC ya make is running...
🔴 2024-11-04 14:02:33 UTC Build failed, see the logs. Also see fail summary

github-actions · 2024-11-04T13:45:40Z

⚪ 2024-11-04 13:45:40 UTC Pre-commit check linux-x86_64-release-asan for be78ed0 has started.
⚪ 2024-11-04 13:45:50 UTC Artifacts will be uploaded here
⚪ 2024-11-04 13:48:57 UTC ya make is running...
🔴 2024-11-04 14:28:29 UTC Build failed, see the logs. Also see fail summary
🔴 2024-11-04 14:29:03 UTC ydbd size 5.7 GiB changed* by +6.2 MiB, which is >= 2.0 MiB vs main: Alert

ydbd size dash	main: `f12f073`	merge: `be78ed0`	diff	diff %
ydbd size	6 147 342 800 Bytes	6 153 809 176 Bytes	+6.2 MiB	+0.105%
ydbd stripped size	1 534 493 616 Bytes	1 535 175 504 Bytes	+665.9 KiB	+0.044%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-11-04T15:01:49Z

⚪ 2024-11-04 15:01:48 UTC Pre-commit check linux-x86_64-release-asan for 7e91b59 has started.
⚪ 2024-11-04 15:01:59 UTC Artifacts will be uploaded here
⚪ 2024-11-04 15:05:05 UTC ya make is running...
🟡 2024-11-04 16:26:00 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
9681	9468	0	123	6	84

🟢 2024-11-04 16:26:54 UTC Build successful.
🔴 2024-11-04 16:27:27 UTC ydbd size 5.7 GiB changed* by +6.2 MiB, which is >= 2.0 MiB vs main: Alert

ydbd size dash	main: `f12f073`	merge: `7e91b59`	diff	diff %
ydbd size	6 147 342 800 Bytes	6 153 809 176 Bytes	+6.2 MiB	+0.105%
ydbd stripped size	1 534 493 616 Bytes	1 535 175 504 Bytes	+665.9 KiB	+0.044%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-11-04T15:01:49Z

⚪ 2024-11-04 15:01:48 UTC Pre-commit check linux-x86_64-relwithdebinfo for 7e91b59 has started.
⚪ 2024-11-04 15:02:00 UTC Artifacts will be uploaded here
⚪ 2024-11-04 15:05:18 UTC ya make is running...
🟡 2024-11-04 16:07:40 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
40621	34954	0	8	5558	101

⚪ 2024-11-04 16:11:11 UTC ya make is running... (failed tests rerun, try 2)
🟡 2024-11-04 16:22:53 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
105 (only retried tests)	13	0	1	0	91

⚪ 2024-11-04 16:23:04 UTC ya make is running... (failed tests rerun, try 3)
🔴 2024-11-04 16:42:33 UTC Some tests failed, follow the links below.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
92 (only retried tests)	0	0	1	0	91

🟢 2024-11-04 16:42:40 UTC Build successful.
🔴 2024-11-04 16:43:04 UTC ydbd size 2.8 GiB changed* by +3.0 MiB, which is >= 2.0 MiB vs main: Alert

ydbd size dash	main: `f12f073`	merge: `7e91b59`	diff	diff %
ydbd size	3 038 072 456 Bytes	3 041 168 888 Bytes	+3.0 MiB	+0.102%
ydbd stripped size	481 429 176 Bytes	481 597 752 Bytes	+164.6 KiB	+0.035%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-11-05T14:54:43Z

⚪ 2024-11-05 14:54:43 UTC Pre-commit check linux-x86_64-relwithdebinfo for 98ceb77 has started.
⚪ 2024-11-05 14:54:54 UTC Artifacts will be uploaded here
⚪ 2024-11-05 14:57:59 UTC ya make is running...
🟡 2024-11-05 15:59:21 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
40641	34970	0	16	5557	98

⚪ 2024-11-05 16:02:50 UTC ya make is running... (failed tests rerun, try 2)
🟡 2024-11-05 16:15:04 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
113 (only retried tests)	4	0	14	0	95

⚪ 2024-11-05 16:15:12 UTC ya make is running... (failed tests rerun, try 3)
🔴 2024-11-05 16:30:49 UTC Some tests failed, follow the links below.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
109 (only retried tests)	0	0	14	0	95

🟢 2024-11-05 16:30:56 UTC Build successful.
🔴 2024-11-05 16:31:17 UTC ydbd size 2.8 GiB changed* by +3.3 MiB, which is >= 2.0 MiB vs main: Alert

ydbd size dash	main: `965f473`	merge: `98ceb77`	diff	diff %
ydbd size	3 039 616 488 Bytes	3 043 031 024 Bytes	+3.3 MiB	+0.112%
ydbd stripped size	481 640 376 Bytes	481 820 152 Bytes	+175.6 KiB	+0.037%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-11-05T14:55:24Z

⚪ 2024-11-05 14:55:24 UTC Pre-commit check linux-x86_64-release-asan for 98ceb77 has started.
⚪ 2024-11-05 14:56:23 UTC Artifacts will be uploaded here
⚪ 2024-11-05 15:00:14 UTC ya make is running...
🟡 2024-11-05 16:32:06 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
9681	9463	0	126	9	83

🟢 2024-11-05 16:32:57 UTC Build successful.
🔴 2024-11-05 16:33:27 UTC ydbd size 5.7 GiB changed* by +6.7 MiB, which is >= 2.0 MiB vs main: Alert

ydbd size dash	main: `965f473`	merge: `98ceb77`	diff	diff %
ydbd size	6 150 742 616 Bytes	6 157 743 256 Bytes	+6.7 MiB	+0.114%
ydbd stripped size	1 535 211 088 Bytes	1 535 936 144 Bytes	+708.1 KiB	+0.047%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-11-06T11:09:46Z

⚪ 2024-11-06 11:09:45 UTC Pre-commit check linux-x86_64-release-asan for b7c5852 has started.
⚪ 2024-11-06 11:09:56 UTC Artifacts will be uploaded here
⚪ 2024-11-06 11:13:10 UTC ya make is running...
🟡 2024-11-06 12:35:50 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
9636	9545	0	40	14	37

🟢 2024-11-06 12:36:51 UTC Build successful.
🔴 2024-11-06 12:37:24 UTC ydbd size 5.7 GiB changed* by +6.7 MiB, which is >= 2.0 MiB vs main: Alert

ydbd size dash	main: `849baa4`	merge: `b7c5852`	diff	diff %
ydbd size	6 153 428 928 Bytes	6 160 447 760 Bytes	+6.7 MiB	+0.114%
ydbd stripped size	1 535 572 688 Bytes	1 536 305 872 Bytes	+716.0 KiB	+0.048%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-11-06T11:10:22Z

⚪ 2024-11-06 11:10:21 UTC Pre-commit check linux-x86_64-relwithdebinfo for b7c5852 has started.
⚪ 2024-11-06 11:11:06 UTC Artifacts will be uploaded here
⚪ 2024-11-06 11:14:35 UTC ya make is running...
🟡 2024-11-06 12:14:41 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
40654	34995	0	3	5557	99

⚪ 2024-11-06 12:18:11 UTC ya make is running... (failed tests rerun, try 2)
🟢 2024-11-06 12:35:17 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
102 (only retried tests)	6	0	0	1	95

🟢 2024-11-06 12:35:24 UTC Build successful.
🟠 2024-11-06 12:35:36 UTC Muted new 4 tests
🔴 2024-11-06 12:35:54 UTC ydbd size 2.8 GiB changed* by +3.3 MiB, which is >= 2.0 MiB vs main: Alert

ydbd size dash	main: `849baa4`	merge: `b7c5852`	diff	diff %
ydbd size	3 040 923 984 Bytes	3 044 347 008 Bytes	+3.3 MiB	+0.113%
ydbd stripped size	481 720 952 Bytes	481 904 888 Bytes	+179.6 KiB	+0.038%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

Conflicts: .github/config/muted_ya.txt ydb/core/tx/columnshard/columnshard__init.cpp

Conflicts: .github/config/muted_ya.txt

github-actions bot added the improvement label Nov 4, 2024

ivanmorozov333 added 23 commits November 6, 2024 13:57

fetch data accessors for change tasks

e573166

fixes

4dc8788

fixes

12d82c1

fixes

d131170

fix

ffd3011

fix

640b900

fix build

d98f4ad

fix

d9d16d6

fix

0181872

fix

2233076

actor for sync

f3490de

corrections

2771a97

corrections for sys_view chunks

27e1831

fixes

336d371

fixes

011b6a7

fix

1e3ea08

fix tests

171fca6

corrections

32a108c

fixes

fd06674

local db loading virtualization

e811ff4

fix

d1a6908

corrections

a962394

fix

b707898

ivanmorozov333 added 6 commits November 6, 2024 14:03

fix

c9b065e

simplifications for restore logic in future

d498398

fix

f16cf9c

fixes

f312955

mute

51d7c32

fix loading

9ca70bd

ivanmorozov333 force-pushed the async_fetch_portion_data_access_info branch from a0b2b4a to 9ca70bd Compare November 6, 2024 11:05

fix

eb6bb78

ivanmorozov333 marked this pull request as ready for review November 6, 2024 12:53

ivanmorozov333 requested a review from a team as a code owner November 6, 2024 12:53

zverevgeny approved these changes Nov 6, 2024

View reviewed changes

ivanmorozov333 merged commit dbd5f95 into ydb-platform:main Nov 6, 2024
10 checks passed

ivanmorozov333 deleted the async_fetch_portion_data_access_info branch November 6, 2024 12:54

GrigoriyPA mentioned this pull request Nov 6, 2024

YQ RD support strings parsing in Json columns #11331

Merged

niksaveliev mentioned this pull request Nov 8, 2024

Fix read session correct close ut #11407

Merged

This was referenced Nov 10, 2024

YQ RD enable LLVM in purecalc filters #11442

Merged

YQ WM fixed race with actor context #11576

Merged

niksaveliev mentioned this pull request Nov 14, 2024

Commit for autopartitioned topics #11629

Open

zverevgeny pushed a commit to zverevgeny/ydb that referenced this pull request Jan 2, 2025

Async fetch portion data access info (ydb-platform#11246)

16bfe52

Conflicts: .github/config/muted_ya.txt ydb/core/tx/columnshard/columnshard__init.cpp

zverevgeny pushed a commit to zverevgeny/ydb that referenced this pull request Jan 5, 2025

Async fetch portion data access info (ydb-platform#11246)

9ce734c

Conflicts: .github/config/muted_ya.txt ydb/core/tx/columnshard/columnshard__init.cpp

zverevgeny pushed a commit to zverevgeny/ydb that referenced this pull request Jan 5, 2025

Async fetch portion data access info (ydb-platform#11246)

7a60c7e

Conflicts: .github/config/muted_ya.txt ydb/core/tx/columnshard/columnshard__init.cpp

zverevgeny pushed a commit to zverevgeny/ydb that referenced this pull request Jan 8, 2025

Async fetch portion data access info (ydb-platform#11246)

98cc555

Conflicts: .github/config/muted_ya.txt

zverevgeny mentioned this pull request Jan 9, 2025

Integrate changes from main #13202

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Async fetch portion data access info #11246

Async fetch portion data access info #11246

ivanmorozov333 commented Nov 4, 2024

github-actions bot commented Nov 4, 2024 •

edited

Loading

github-actions bot commented Nov 4, 2024 •

edited

Loading

github-actions bot commented Nov 4, 2024 •

edited

Loading

github-actions bot commented Nov 4, 2024 •

edited

Loading

github-actions bot commented Nov 5, 2024 •

edited

Loading

github-actions bot commented Nov 5, 2024 •

edited

Loading

github-actions bot commented Nov 6, 2024 •

edited

Loading

github-actions bot commented Nov 6, 2024 •

edited

Loading

Async fetch portion data access info #11246

Async fetch portion data access info #11246

Conversation

ivanmorozov333 commented Nov 4, 2024

github-actions bot commented Nov 4, 2024 • edited Loading

github-actions bot commented Nov 4, 2024 • edited Loading

github-actions bot commented Nov 4, 2024 • edited Loading

github-actions bot commented Nov 4, 2024 • edited Loading

github-actions bot commented Nov 5, 2024 • edited Loading

github-actions bot commented Nov 5, 2024 • edited Loading

github-actions bot commented Nov 6, 2024 • edited Loading

github-actions bot commented Nov 6, 2024 • edited Loading

github-actions bot commented Nov 4, 2024 •

edited

Loading

github-actions bot commented Nov 4, 2024 •

edited

Loading

github-actions bot commented Nov 4, 2024 •

edited

Loading

github-actions bot commented Nov 4, 2024 •

edited

Loading

github-actions bot commented Nov 5, 2024 •

edited

Loading

github-actions bot commented Nov 5, 2024 •

edited

Loading

github-actions bot commented Nov 6, 2024 •

edited

Loading

github-actions bot commented Nov 6, 2024 •

edited

Loading