Support Put and Get requests acceleration for 2 slow disks in block-4-2 erasure #4803

serbel324 · 2024-05-23T18:17:08Z

Add Acceleration tests
Get Acceleration test
Support Put acceleration for block-4-2

github-actions · 2024-05-23T18:20:29Z

⚪ 2024-05-23 18:20:29 UTC Pre-commit check for 7fbf618 has started.
⚪ 2024-05-23 18:23:01 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-05-23 18:36:51 UTC Build successful.

github-actions · 2024-05-23T18:20:37Z

⚪ 2024-05-23 18:20:37 UTC Pre-commit check for 7fbf618 has started.
⚪ 2024-05-23 18:23:11 UTC Build linux-x86_64-release-asan is running...
🟢 2024-05-23 18:26:46 UTC Build successful.
⚪ 2024-05-23 18:27:00 UTC Tests are running...
🔴 2024-05-23 20:23:46 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
9666	9602	0	26	27	11

github-actions · 2024-05-23T18:20:38Z

⚪ 2024-05-23 18:20:38 UTC Pre-commit check for 7fbf618 has started.
⚪ 2024-05-23 18:23:13 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-05-23 18:27:43 UTC Build successful.
⚪ 2024-05-23 18:27:58 UTC Tests are running...
🔴 2024-05-23 19:42:44 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
13789	12483	0	1	1299	6

github-actions · 2024-05-24T04:17:44Z

⚪ 2024-05-24 04:17:43 UTC Pre-commit check for 0f3d625 has started.
⚪ 2024-05-24 04:20:16 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-05-24 04:22:10 UTC Build successful.

github-actions · 2024-05-24T04:17:48Z

⚪ 2024-05-24 04:17:48 UTC Pre-commit check for 0f3d625 has started.
⚪ 2024-05-24 04:20:26 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-05-24 04:22:25 UTC Build successful.
⚪ 2024-05-24 04:22:40 UTC Tests are running...
🔴 2024-05-24 05:40:12 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
13789	12482	0	1	1299	7

github-actions · 2024-05-24T04:19:49Z

⚪ 2024-05-24 04:19:49 UTC Pre-commit check for 0f3d625 has started.
⚪ 2024-05-24 04:22:18 UTC Build linux-x86_64-release-asan is running...
🟢 2024-05-24 04:24:38 UTC Build successful.
⚪ 2024-05-24 04:24:50 UTC Tests are running...
🔴 2024-05-24 06:15:24 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
9654	9597	0	21	22	14

ydb/core/blobstorage/dsproxy/dsproxy_blackboard.cpp

ydb/core/blobstorage/dsproxy/dsproxy_put.cpp

alexvru · 2024-05-24T13:32:53Z

ydb/core/blobstorage/dsproxy/dsproxy_put.cpp

        Action(true);
 //        *(IsMultiPutMode ? Mon->NodeMon->AccelerateEvVMultiPutCount : Mon->NodeMon->AccelerateEvVPutCount) += v.size();
+        AccelerateIfNeeded();


Action(true) a few lines above already accelerates, no?

Yes, but we need to schedule second accelerate in case current acceleration sends request to slow disk and doesn't recieve response in time. This call schedules next acceleration.

github-actions · 2024-05-24T15:56:15Z

⚪ 2024-05-24 15:56:15 UTC Pre-commit check for b08f4da has started.
⚪ 2024-05-24 15:58:37 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-05-24 16:30:46 UTC Build successful.

github-actions · 2024-05-24T15:56:46Z

⚪ 2024-05-24 15:56:45 UTC Pre-commit check for b08f4da has started.
⚪ 2024-05-24 15:59:10 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-05-24 16:14:54 UTC Build successful.
⚪ 2024-05-24 16:15:06 UTC Tests are running...
🔴 2024-05-24 17:30:17 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
13804	11406	0	1044	1320	34

github-actions · 2024-05-24T16:18:24Z

⚪ 2024-05-24 16:18:24 UTC Pre-commit check for b08f4da has started.
⚪ 2024-05-24 16:20:49 UTC Build linux-x86_64-release-asan is running...
🟢 2024-05-24 16:23:48 UTC Build successful.
⚪ 2024-05-24 16:24:01 UTC Tests are running...
🔴 2024-05-24 18:15:21 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
9672	8574	0	1010	45	43

github-actions · 2024-05-27T12:35:29Z

⚪ 2024-05-27 12:35:29 UTC Pre-commit check for d1ec482 has started.
⚪ 2024-05-27 12:37:54 UTC Build linux-x86_64-release-asan is running...
🟢 2024-05-27 12:40:49 UTC Build successful.
⚪ 2024-05-27 12:40:58 UTC Tests are running...
🔴 2024-05-27 14:34:53 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
9672	9597	0	28	30	17

github-actions · 2024-05-27T12:39:46Z

⚪ 2024-05-27 12:39:46 UTC Pre-commit check for d1ec482 has started.
⚪ 2024-05-27 12:42:07 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-05-27 12:44:25 UTC Build successful.

github-actions · 2024-05-27T12:44:48Z

⚪ 2024-05-27 12:44:48 UTC Pre-commit check for d1ec482 has started.
⚪ 2024-05-27 12:47:09 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-05-27 12:49:36 UTC Build successful.
⚪ 2024-05-27 12:49:46 UTC Tests are running...
🔴 2024-05-27 14:06:09 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
13800	12484	0	11	1299	6

github-actions · 2024-05-28T14:24:08Z

⚪ 2024-05-28 14:24:07 UTC Pre-commit check for ba1c745 has started.
⚪ 2024-05-28 14:26:25 UTC Build linux-x86_64-release-asan is running...
🟢 2024-05-28 14:29:07 UTC Build successful.
⚪ 2024-05-28 14:29:21 UTC Tests are running...
🔴 2024-05-28 16:19:21 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
9664	9607	0	23	21	13

github-actions · 2024-05-28T14:28:17Z

⚪ 2024-05-28 14:28:17 UTC Pre-commit check for ba1c745 has started.
⚪ 2024-05-28 14:30:46 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-05-28 14:33:21 UTC Build successful.
⚪ 2024-05-28 14:33:30 UTC Tests are running...
🔴 2024-05-28 15:47:57 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
13831	12495	0	6	1320	10

github-actions · 2024-05-28T14:30:13Z

⚪ 2024-05-28 14:30:13 UTC Pre-commit check for ba1c745 has started.
⚪ 2024-05-28 14:32:30 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-05-28 14:35:51 UTC Build successful.

github-actions · 2024-05-31T15:07:17Z

⚪ 2024-05-31 15:07:17 UTC Pre-commit check for 14a6867 has started.
⚪ 2024-05-31 15:09:43 UTC Build linux-x86_64-release-asan is running...
🟢 2024-05-31 15:12:34 UTC Build successful.
⚪ 2024-05-31 15:12:49 UTC Tests are running...
🔴 2024-05-31 16:49:39 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
9510	9430	0	34	32	14

github-actions · 2024-05-31T15:18:01Z

⚪ 2024-05-31 15:18:01 UTC Pre-commit check for 14a6867 has started.
⚪ 2024-05-31 15:20:25 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-05-31 15:22:55 UTC Build successful.

github-actions · 2024-05-31T15:19:08Z

⚪ 2024-05-31 15:19:08 UTC Pre-commit check for 14a6867 has started.
⚪ 2024-05-31 15:21:34 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-05-31 15:24:02 UTC Build successful.
⚪ 2024-05-31 15:24:12 UTC Tests are running...
🔴 2024-05-31 16:32:31 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
13671	12335	0	13	1308	15

serbel324 added 3 commits May 16, 2024 11:10

Add Acceleration tests

ada6ecc

Get Acceleration test

41b92dc

Support Put acceleration for block-4-2

219ef6e

Temporarily mute Test4Plus2BlockGetAsyncRead2Slow size

80ebaca

alexvru reviewed May 24, 2024

View reviewed changes

Add double acceleration for get requests

ddf5151

serbel324 changed the title ~~Support Put acceleration for 2 slow disks in block-4-2 erasure~~ Support Put and Get requests acceleration for 2 slow disks in block-4-2 erasure May 24, 2024

Address comments

54432e6

serbel324 added 2 commits May 27, 2024 09:51

DEBUG verify

4855ebd

Fix acceleration with ErasureNone

fb9369f

Revert splitting RequestsSent and ResponsesReceived

8f40cb4

Fix sorting

cc6b495

serbel324 linked an issue May 31, 2024 that may be closed by this pull request

Accelerate dsproxy requests when 2 slow disks are present, block-4-2 #4828

Closed

alexvru approved these changes Jun 4, 2024

View reviewed changes

serbel324 merged commit af3b3b4 into ydb-platform:main Jun 5, 2024
8 of 12 checks passed

This was referenced Jun 6, 2024

Mute split by load ut #5264

Closed

Mute split by load ut #5267

Closed

Mute split by load ut #5272

Merged

Fix build #5310

Merged

serbel324 mentioned this pull request Jun 7, 2024

Don't mark slow disks if there is only one #5340

Merged

GrigoriyPA mentioned this pull request Jun 10, 2024

KqpRun fixed performance and backtrace printing #5368

Merged

niksaveliev mentioned this pull request Jun 10, 2024

Fix split by load ut #5383

Merged

This was referenced Jun 11, 2024

metadata cache keeps merged issues #5461

Merged

json_hc metadata cache #5465

Merged

GrigoriyPA mentioned this pull request Jun 13, 2024

YQ-3333 added yql syntax for creating resource pools #5498

Merged

This was referenced Jun 14, 2024

json tenantinfo status metadata cache #5543

Merged

json query transaction mode #5574

Merged

This was referenced Jun 15, 2024

YQ-3225 detached script execution results forgetting #5598

Merged

YQ-3346 improve secrets fetching #5614

Merged

niksaveliev mentioned this pull request Jun 17, 2024

Fix win build #5622

Closed

GrigoriyPA mentioned this pull request Jun 18, 2024

YQ-3342 fix script results writing #5682

Merged

StekPerepolnen mentioned this pull request Jun 19, 2024

hc api doc #5703

Merged

This was referenced Jun 19, 2024

WM passed resource pool settings to user request context #5734

Merged

YQ splitted fq/s3/test_s3.py tests #5798

Merged

This was referenced Jun 23, 2024

move time difference issue under database level #5859

Merged

metadata requests flags fix #5860

Merged

oidc-proxy new iam 2 #6059

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support Put and Get requests acceleration for 2 slow disks in block-4-2 erasure #4803

Support Put and Get requests acceleration for 2 slow disks in block-4-2 erasure #4803

serbel324 commented May 23, 2024

github-actions bot commented May 23, 2024 •

edited

Loading

github-actions bot commented May 23, 2024 •

edited

Loading

github-actions bot commented May 23, 2024 •

edited

Loading

github-actions bot commented May 24, 2024 •

edited

Loading

github-actions bot commented May 24, 2024 •

edited

Loading

github-actions bot commented May 24, 2024 •

edited

Loading

alexvru May 24, 2024

serbel324 May 24, 2024

github-actions bot commented May 24, 2024 •

edited

Loading

github-actions bot commented May 24, 2024 •

edited

Loading

github-actions bot commented May 24, 2024 •

edited

Loading

github-actions bot commented May 27, 2024 •

edited

Loading

github-actions bot commented May 27, 2024 •

edited

Loading

github-actions bot commented May 27, 2024 •

edited

Loading

github-actions bot commented May 28, 2024 •

edited

Loading

github-actions bot commented May 28, 2024 •

edited

Loading

github-actions bot commented May 28, 2024 •

edited

Loading

github-actions bot commented May 31, 2024 •

edited

Loading

github-actions bot commented May 31, 2024 •

edited

Loading

github-actions bot commented May 31, 2024 •

edited

Loading

Support Put and Get requests acceleration for 2 slow disks in block-4-2 erasure #4803

Support Put and Get requests acceleration for 2 slow disks in block-4-2 erasure #4803

Conversation

serbel324 commented May 23, 2024

github-actions bot commented May 23, 2024 • edited Loading

github-actions bot commented May 23, 2024 • edited Loading

github-actions bot commented May 23, 2024 • edited Loading

github-actions bot commented May 24, 2024 • edited Loading

github-actions bot commented May 24, 2024 • edited Loading

github-actions bot commented May 24, 2024 • edited Loading

alexvru May 24, 2024

Choose a reason for hiding this comment

serbel324 May 24, 2024

Choose a reason for hiding this comment

github-actions bot commented May 24, 2024 • edited Loading

github-actions bot commented May 24, 2024 • edited Loading

github-actions bot commented May 24, 2024 • edited Loading

github-actions bot commented May 27, 2024 • edited Loading

github-actions bot commented May 27, 2024 • edited Loading

github-actions bot commented May 27, 2024 • edited Loading

github-actions bot commented May 28, 2024 • edited Loading

github-actions bot commented May 28, 2024 • edited Loading

github-actions bot commented May 28, 2024 • edited Loading

github-actions bot commented May 31, 2024 • edited Loading

github-actions bot commented May 31, 2024 • edited Loading

github-actions bot commented May 31, 2024 • edited Loading

github-actions bot commented May 23, 2024 •

edited

Loading

github-actions bot commented May 23, 2024 •

edited

Loading

github-actions bot commented May 23, 2024 •

edited

Loading

github-actions bot commented May 24, 2024 •

edited

Loading

github-actions bot commented May 24, 2024 •

edited

Loading

github-actions bot commented May 24, 2024 •

edited

Loading

github-actions bot commented May 24, 2024 •

edited

Loading

github-actions bot commented May 24, 2024 •

edited

Loading

github-actions bot commented May 24, 2024 •

edited

Loading

github-actions bot commented May 27, 2024 •

edited

Loading

github-actions bot commented May 27, 2024 •

edited

Loading

github-actions bot commented May 27, 2024 •

edited

Loading

github-actions bot commented May 28, 2024 •

edited

Loading

github-actions bot commented May 28, 2024 •

edited

Loading

github-actions bot commented May 28, 2024 •

edited

Loading

github-actions bot commented May 31, 2024 •

edited

Loading

github-actions bot commented May 31, 2024 •

edited

Loading

github-actions bot commented May 31, 2024 •

edited

Loading