YQL-18314 use shared alloc in all task runners #4233

zverevgeny · 2024-05-02T12:59:30Z

No description provided.

zverevgeny · 2024-05-02T13:01:45Z

ydb/library/yql/providers/dq/task_runner/tasks_runner_pipe.cpp

@@ -1299,8 +1300,7 @@ class TTaskRunner: public IPipeTaskRunner {
        : TraceId(traceId)
        , Task(task)
        , FilesHolder(std::move(filesHolder))
-        , Alloc(new NKikimr::NMiniKQL::TScopedAlloc(__LOCATION__, NKikimr::TAlignedPagePoolCounters(), true),


this is the problem being fixed

github-actions · 2024-05-02T13:02:48Z

⚪ 2024-05-02 13:02:48 UTC Pre-commit check for 5135299 has started.
⚪ 2024-05-02 13:02:51 UTC Build linux-x86_64-release-clang14 is running...
🔴 2024-05-02 13:12:06 UTC Build failed. see the build logs.

github-actions · 2024-05-02T13:02:55Z

⚪ 2024-05-02 13:02:55 UTC Pre-commit check for 5135299 has started.
⚪ 2024-05-02 13:02:57 UTC Build linux-x86_64-release-asan is running...
🔴 2024-05-02 13:13:15 UTC Build failed. see the build logs.
🔴 2024-05-02 13:14:50 UTC Tests run skipped.

github-actions · 2024-05-02T13:03:01Z

⚪ 2024-05-02 13:03:01 UTC Pre-commit check for 5135299 has started.
⚪ 2024-05-02 13:03:03 UTC Build linux-x86_64-relwithdebinfo is running...
🔴 2024-05-02 13:12:50 UTC Build failed. see the build logs.
🔴 2024-05-02 13:14:28 UTC Tests run skipped.

github-actions · 2024-05-02T14:22:41Z

⚪ 2024-05-02 14:22:41 UTC Pre-commit check for b998bbe has started.
⚪ 2024-05-02 14:22:42 UTC Build linux-x86_64-relwithdebinfo is running...
🔴 2024-05-02 14:24:18 UTC Build failed. see the build logs.
🔴 2024-05-02 14:25:46 UTC Tests run skipped.

github-actions · 2024-05-02T14:23:15Z

⚪ 2024-05-02 14:23:14 UTC Pre-commit check for b998bbe has started.
⚪ 2024-05-02 14:23:16 UTC Build linux-x86_64-release-asan is running...
🔴 2024-05-02 14:24:46 UTC Build failed. see the build logs.
🔴 2024-05-02 14:26:11 UTC Tests run skipped.

github-actions · 2024-05-02T14:23:36Z

⚪ 2024-05-02 14:23:36 UTC Pre-commit check for b998bbe has started.
⚪ 2024-05-02 14:23:38 UTC Build linux-x86_64-release-clang14 is running...
🔴 2024-05-02 14:25:28 UTC Build failed. see the build logs.

github-actions · 2024-05-02T14:40:10Z

⚪ 2024-05-02 14:40:10 UTC Pre-commit check for 95052a5 has started.
⚪ 2024-05-02 14:40:12 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-05-02 14:41:44 UTC Build successful.

github-actions · 2024-05-02T14:40:14Z

⚪ 2024-05-02 14:40:14 UTC Pre-commit check for 95052a5 has started.
⚪ 2024-05-02 14:40:16 UTC Build linux-x86_64-release-asan is running...
🟢 2024-05-02 14:41:42 UTC Build successful.
⚪ 2024-05-02 14:43:28 UTC Tests are running...
🔴 2024-05-02 15:02:04 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
4064	3799	0	241	19	5

github-actions · 2024-05-02T14:40:15Z

⚪ 2024-05-02 14:40:15 UTC Pre-commit check for 95052a5 has started.
⚪ 2024-05-02 14:40:17 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-05-02 14:41:50 UTC Build successful.
⚪ 2024-05-02 14:43:38 UTC Tests are running...
🔴 2024-05-02 15:35:51 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
58838	47323	0	257	11253	5

rvu1024 · 2024-05-02T15:28:08Z

ydb/core/kqp/runtime/kqp_tasks_runner.h


    const TMap<ui64, const NYql::NDq::TDqTaskRunnerStats*> GetTasksStats() const { return Stats; }
 private:
    TMap<ui64, TIntrusivePtr<NYql::NDq::IDqTaskRunner>> TaskRunners;
    TMap<ui64, NYql::NDq::TDqTaskSettings> Tasks;
    TMap<ui64, const NYql::NDq::TDqTaskRunnerStats*> Stats;
    NYql::NDq::TLogFunc LogFunc;
-    NMiniKQL::TScopedAlloc& Alloc;
+   std::shared_ptr<NKikimr::NMiniKQL::TScopedAlloc> Alloc;


indent кажется поплыл

github-actions · 2024-05-03T03:18:15Z

⚪ 2024-05-03 03:18:14 UTC Pre-commit check for cbc23c8 has started.
⚪ 2024-05-03 03:18:18 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-05-03 03:26:37 UTC Build successful.

github-actions · 2024-05-03T03:18:37Z

⚪ 2024-05-03 03:18:36 UTC Pre-commit check for cbc23c8 has started.
⚪ 2024-05-03 03:18:39 UTC Build linux-x86_64-release-asan is running...
🟢 2024-05-03 03:27:00 UTC Build successful.
⚪ 2024-05-03 03:28:48 UTC Tests are running...
🔴 2024-05-03 03:47:45 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
4063	3803	0	236	19	5

github-actions · 2024-05-03T03:21:31Z

⚪ 2024-05-03 03:21:31 UTC Pre-commit check for cbc23c8 has started.
⚪ 2024-05-03 03:21:33 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-05-03 03:30:12 UTC Build successful.
⚪ 2024-05-03 03:32:03 UTC Tests are running...
🔴 2024-05-03 04:24:17 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
58839	47324	0	257	11253	5

github-actions · 2024-05-03T06:33:51Z

⚪ 2024-05-03 06:33:51 UTC Pre-commit check for 39b4b07 has started.
⚪ 2024-05-03 06:33:54 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-05-03 06:45:40 UTC Build successful.

github-actions · 2024-05-03T06:34:18Z

⚪ 2024-05-03 06:34:18 UTC Pre-commit check for 39b4b07 has started.
⚪ 2024-05-03 06:34:21 UTC Build linux-x86_64-release-asan is running...
🟢 2024-05-03 06:43:50 UTC Build successful.
⚪ 2024-05-03 06:45:37 UTC Tests are running...
🔴 2024-05-03 09:03:53 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
12005	9037	0	2463	356	149

github-actions · 2024-05-03T06:34:52Z

⚪ 2024-05-03 06:34:52 UTC Pre-commit check for 39b4b07 has started.
⚪ 2024-05-03 06:34:55 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-05-03 06:45:00 UTC Build successful.
⚪ 2024-05-03 06:46:54 UTC Tests are running...
🔴 2024-05-03 09:12:18 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
70638	52844	0	3691	13952	151

github-actions · 2024-05-27T16:30:42Z

⚪ 2024-05-27 16:30:42 UTC Pre-commit check for f401cdf has started.
⚪ 2024-05-27 16:33:16 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-05-27 16:42:12 UTC Build successful.
⚪ 2024-05-27 16:42:26 UTC Tests are running...
🔴 2024-05-27 18:36:53 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
71501	58783	0	11	12702	5

github-actions · 2024-05-28T03:32:52Z

⚪ 2024-05-28 03:32:52 UTC Pre-commit check for b02b1c7 has started.
⚪ 2024-05-28 03:35:35 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-05-28 03:38:58 UTC Build successful.
⚪ 2024-05-28 03:39:12 UTC Tests are running...
🔴 2024-05-28 05:34:33 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
71519	58805	0	5	12704	5

zverevgeny requested review from a team as code owners May 2, 2024 12:59

zverevgeny commented May 2, 2024

View reviewed changes

zverevgeny requested a review from gridnevvvit May 2, 2024 13:05

resetius previously approved these changes May 2, 2024

View reviewed changes

zverevgeny dismissed resetius’s stale review via febd236 May 2, 2024 14:20

zverevgeny requested a review from a team as a code owner May 2, 2024 14:20

gridnevvvit previously approved these changes May 2, 2024

View reviewed changes

rvu1024 reviewed May 2, 2024

View reviewed changes

lll-phill-lll previously approved these changes May 2, 2024

View reviewed changes

zverevgeny dismissed stale reviews from lll-phill-lll and gridnevvvit via b6be2cb May 3, 2024 03:15

zverevgeny force-pushed the YQL-18314_shared_alloc_in_task_runner branch from 0114bce to b6be2cb Compare May 3, 2024 03:15

zverevgeny force-pushed the YQL-18314_shared_alloc_in_task_runner branch from e21e733 to 4d34664 Compare May 27, 2024 16:27

rvu1024 approved these changes May 28, 2024

View reviewed changes

uzhastik approved these changes May 28, 2024

View reviewed changes

zverevgeny merged commit 4e3063e into ydb-platform:main May 28, 2024
8 of 12 checks passed

niksaveliev mentioned this pull request May 29, 2024

Autoscaling logs #4976

Merged

This was referenced May 29, 2024

YQ-3292 fixed kqprun ResourceBrokerConfig passing #4994

Merged

Added ExecutionId for script execution logs #5014

Merged

niksaveliev mentioned this pull request May 30, 2024

Forbid autoscaling with size retention #5054

Merged

GrigoriyPA mentioned this pull request May 31, 2024

Fixed build error in TForgetScriptExecutionOperationQueryActor #5084

Merged

StekPerepolnen mentioned this pull request May 31, 2024

vdisk evict #5093

Merged

GrigoriyPA mentioned this pull request Jun 3, 2024

Fixed backward compatibility in fq-proxy result_writer_actor #5111

Merged

This was referenced Jun 6, 2024

Mute split by load ut #5264

Closed

Mute split by load ut #5267

Closed

Mute split by load ut #5272

Merged

Fix build #5310

Merged

serbel324 mentioned this pull request Jun 7, 2024

Don't mark slow disks if there is only one #5340

Merged

GrigoriyPA mentioned this pull request Jun 10, 2024

KqpRun fixed performance and backtrace printing #5368

Merged

niksaveliev mentioned this pull request Jun 10, 2024

Fix split by load ut #5383

Merged

This was referenced Jun 11, 2024

metadata cache keeps merged issues #5461

Merged

json_hc metadata cache #5465

Merged

GrigoriyPA mentioned this pull request Jun 13, 2024

YQ-3333 added yql syntax for creating resource pools #5498

Merged

This was referenced Jun 14, 2024

json tenantinfo status metadata cache #5543

Merged

json query transaction mode #5574

Merged

niksaveliev mentioned this pull request Jun 17, 2024

Fix win build #5622

Closed

StekPerepolnen mentioned this pull request Jun 19, 2024

hc api doc #5703

Merged

This was referenced Jun 19, 2024

WM passed resource pool settings to user request context #5734

Merged

YQ splitted fq/s3/test_s3.py tests #5798

Merged

This was referenced Jun 23, 2024

move time difference issue under database level #5859

Merged

metadata requests flags fix #5860

Merged

oidc-proxy new iam 2 #6059

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

YQL-18314 use shared alloc in all task runners #4233

YQL-18314 use shared alloc in all task runners #4233

zverevgeny commented May 2, 2024

zverevgeny May 2, 2024

github-actions bot commented May 2, 2024 •

edited

Loading

github-actions bot commented May 2, 2024 •

edited

Loading

github-actions bot commented May 2, 2024 •

edited

Loading

github-actions bot commented May 2, 2024 •

edited

Loading

github-actions bot commented May 2, 2024 •

edited

Loading

github-actions bot commented May 2, 2024 •

edited

Loading

github-actions bot commented May 2, 2024 •

edited

Loading

github-actions bot commented May 2, 2024 •

edited

Loading

github-actions bot commented May 2, 2024 •

edited

Loading

rvu1024 May 2, 2024

github-actions bot commented May 3, 2024 •

edited

Loading

github-actions bot commented May 3, 2024 •

edited

Loading

github-actions bot commented May 3, 2024 •

edited

Loading

github-actions bot commented May 3, 2024 •

edited

Loading

github-actions bot commented May 3, 2024 •

edited

Loading

github-actions bot commented May 3, 2024 •

edited

Loading

github-actions bot commented May 27, 2024 •

edited

Loading

github-actions bot commented May 28, 2024 •

edited

Loading

YQL-18314 use shared alloc in all task runners #4233

YQL-18314 use shared alloc in all task runners #4233

Conversation

zverevgeny commented May 2, 2024

zverevgeny May 2, 2024

Choose a reason for hiding this comment

github-actions bot commented May 2, 2024 • edited Loading

github-actions bot commented May 2, 2024 • edited Loading

github-actions bot commented May 2, 2024 • edited Loading

github-actions bot commented May 2, 2024 • edited Loading

github-actions bot commented May 2, 2024 • edited Loading

github-actions bot commented May 2, 2024 • edited Loading

github-actions bot commented May 2, 2024 • edited Loading

github-actions bot commented May 2, 2024 • edited Loading

github-actions bot commented May 2, 2024 • edited Loading

rvu1024 May 2, 2024

Choose a reason for hiding this comment

github-actions bot commented May 3, 2024 • edited Loading

github-actions bot commented May 3, 2024 • edited Loading

github-actions bot commented May 3, 2024 • edited Loading

github-actions bot commented May 3, 2024 • edited Loading

github-actions bot commented May 3, 2024 • edited Loading

github-actions bot commented May 3, 2024 • edited Loading

github-actions bot commented May 27, 2024 • edited Loading

github-actions bot commented May 28, 2024 • edited Loading

github-actions bot commented May 2, 2024 •

edited

Loading

github-actions bot commented May 2, 2024 •

edited

Loading

github-actions bot commented May 2, 2024 •

edited

Loading

github-actions bot commented May 2, 2024 •

edited

Loading

github-actions bot commented May 2, 2024 •

edited

Loading

github-actions bot commented May 2, 2024 •

edited

Loading

github-actions bot commented May 2, 2024 •

edited

Loading

github-actions bot commented May 2, 2024 •

edited

Loading

github-actions bot commented May 2, 2024 •

edited

Loading

github-actions bot commented May 3, 2024 •

edited

Loading

github-actions bot commented May 3, 2024 •

edited

Loading

github-actions bot commented May 3, 2024 •

edited

Loading

github-actions bot commented May 3, 2024 •

edited

Loading

github-actions bot commented May 3, 2024 •

edited

Loading

github-actions bot commented May 3, 2024 •

edited

Loading

github-actions bot commented May 27, 2024 •

edited

Loading

github-actions bot commented May 28, 2024 •

edited

Loading