YQL-18052: Introduce CLI option to purebench to toggle block engine mode #4526

igormunkin · 2024-05-14T18:56:16Z

This patchset introduces CLI option (-b/--block-engine) to purebench for toggling block engine mode. It is initially set as a string value (one from 'force', 'disable' and 'auto'), and converted to EBlockEngineMode in program factory constructor. If the conversion fails, TCompileError is raised.

There was also a tiny bug: the TCompileError error body was not printed. As a result of the first patch of the series this issue is fixed.

Changelog category

New feature

github-actions · 2024-05-14T18:59:15Z

⚪ 2024-05-14 18:59:14 UTC Pre-commit check for 63fc4a8 has started.
⚪ 2024-05-14 18:59:16 UTC Build linux-x86_64-release-clang14 is running...
🔴 2024-05-14 19:01:19 UTC Build failed. see the build logs.

github-actions · 2024-05-14T18:59:44Z

⚪ 2024-05-14 18:59:44 UTC Pre-commit check for 63fc4a8 has started.
⚪ 2024-05-14 18:59:46 UTC Build linux-x86_64-release-asan is running...
🔴 2024-05-14 19:01:35 UTC Build failed. see the build logs.
🔴 2024-05-14 19:02:52 UTC Tests run skipped.

github-actions · 2024-05-14T18:59:49Z

⚪ 2024-05-14 18:59:49 UTC Pre-commit check for 63fc4a8 has started.
⚪ 2024-05-14 18:59:51 UTC Build linux-x86_64-relwithdebinfo is running...
🔴 2024-05-14 19:01:30 UTC Build failed. see the build logs.
🔴 2024-05-14 19:02:47 UTC Tests run skipped.

github-actions · 2024-05-15T14:05:46Z

⚪ 2024-05-15 14:05:45 UTC Pre-commit check for 8bb1edb has started.
⚪ 2024-05-15 14:05:48 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-05-15 14:08:51 UTC Build successful.
⚪ 2024-05-15 14:10:25 UTC Tests are running...
🟢 2024-05-15 14:11:11 UTC Tests successful.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
146	146	0	0	0	0

github-actions · 2024-05-15T14:06:45Z

⚪ 2024-05-15 14:06:45 UTC Pre-commit check for 8bb1edb has started.
⚪ 2024-05-15 14:06:47 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-05-15 14:09:58 UTC Build successful.

github-actions · 2024-05-15T14:07:29Z

⚪ 2024-05-15 14:07:29 UTC Pre-commit check for 8bb1edb has started.
⚪ 2024-05-15 14:07:32 UTC Build linux-x86_64-release-asan is running...
🟢 2024-05-15 14:09:15 UTC Build successful.
⚪ 2024-05-15 14:10:48 UTC Tests are running...
🟢 2024-05-15 14:12:49 UTC Tests successful.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
128	128	0	0	0	0

github-actions · 2024-05-17T11:56:16Z

⚪ 2024-05-17 11:56:16 UTC Pre-commit check for a7483e4 has started.
⚪ 2024-05-17 11:56:18 UTC Build linux-x86_64-release-asan is running...
🟢 2024-05-17 11:57:39 UTC Build successful.
⚪ 2024-05-17 11:59:06 UTC Tests are running...
🟢 2024-05-17 12:00:58 UTC Tests successful.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
128	128	0	0	0	0

github-actions · 2024-05-17T11:58:02Z

⚪ 2024-05-17 11:58:02 UTC Pre-commit check for a7483e4 has started.
⚪ 2024-05-17 11:58:04 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-05-17 12:14:56 UTC Build successful.

github-actions · 2024-05-17T11:58:05Z

⚪ 2024-05-17 11:58:05 UTC Pre-commit check for a7483e4 has started.
⚪ 2024-05-17 11:58:07 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-05-17 12:01:00 UTC Build successful.
⚪ 2024-05-17 12:02:23 UTC Tests are running...
🟢 2024-05-17 12:03:11 UTC Tests successful.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
146	146	0	0	0	0

As a result of the patch, the body of the thrown TCompileError is also yield to stderr.

This patch implements passing through the block engine mode, being set in the program factory, to the worker type context.

This patch introduces BlockEngineSettings option for program factory to tweak the block engine mode. The possible values are "force" to unconditionally enable it, "disable" for turn it off and "auto" to left the final decision to the platform heuristics.

This patch implements conversion from the string option set via the purecalc interface to the corresponding enum value. If the conversion fails, TCompileError is raised. Unfortunately, there is no way to hoist the conversion upper in the call stack, since this will lead to exposure of the private interfaces.

This patch introduces the purebench CLI option to tweak the mode of the block engine while performing the benchmarks.

github-actions · 2024-05-28T11:13:54Z

⚪ 2024-05-28 11:13:54 UTC Pre-commit check for c414eaa has started.
⚪ 2024-05-28 11:15:45 UTC Build linux-x86_64-release-asan is running...
🟢 2024-05-28 11:17:16 UTC Build successful.
⚪ 2024-05-28 11:17:26 UTC Tests are running...
🟢 2024-05-28 11:19:17 UTC Tests successful.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
139	139	0	0	0	0

github-actions · 2024-05-28T11:20:11Z

⚪ 2024-05-28 11:20:10 UTC Pre-commit check for c414eaa has started.
⚪ 2024-05-28 11:22:13 UTC Build linux-x86_64-release-clang14 is running...
🔴 2024-05-28 11:23:51 UTC Build failed. see the build logs.

github-actions · 2024-05-28T11:25:24Z

⚪ 2024-05-28 11:25:24 UTC Pre-commit check for c414eaa has started.
⚪ 2024-05-28 11:27:19 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-05-28 11:28:40 UTC Build successful.
⚪ 2024-05-28 11:28:50 UTC Tests are running...
🟢 2024-05-28 11:29:50 UTC Tests successful.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
159	159	0	0	0	0

github-actions bot added the new-feature label May 14, 2024

igormunkin force-pushed the YQL-18052-add-block-engine-to-purebench branch from d8894df to ed45800 Compare May 15, 2024 14:04

igormunkin force-pushed the YQL-18052-add-block-engine-to-purebench branch from ed45800 to 1db9939 Compare May 17, 2024 11:54

igormunkin added 6 commits May 28, 2024 11:01

Print the body of the thrown TCompileError

f674dce

As a result of the patch, the body of the thrown TCompileError is also yield to stderr.

YQL-18052: Apply block engine mode to purecalc worker

ccb8af1

This patch implements passing through the block engine mode, being set in the program factory, to the worker type context.

YQL-18052: Introduce a CLI option for purebench

717ee28

This patch introduces the purebench CLI option to tweak the mode of the block engine while performing the benchmarks.

Set block engine settings in purecalc example

222bef4

igormunkin force-pushed the YQL-18052-add-block-engine-to-purebench branch from 1db9939 to 222bef4 Compare May 28, 2024 11:08

igormunkin marked this pull request as ready for review May 28, 2024 11:55

igormunkin requested a review from a team as a code owner May 28, 2024 11:55

vitstn approved these changes May 28, 2024

View reviewed changes

igormunkin merged commit 01a89e6 into ydb-platform:main May 28, 2024
11 of 13 checks passed

igormunkin changed the title ~~YQL-18052: Introduce CLI option to purebench to toggle block engine support~~ YQL-18052: Introduce CLI option to purebench to toggle block engine mode May 28, 2024

igormunkin deleted the YQL-18052-add-block-engine-to-purebench branch May 28, 2024 12:14

niksaveliev mentioned this pull request May 29, 2024

Autoscaling logs #4976

Merged

GrigoriyPA mentioned this pull request May 29, 2024

YQ-3292 fixed kqprun ResourceBrokerConfig passing #4994

Merged

GrigoriyPA mentioned this pull request May 30, 2024

Added ExecutionId for script execution logs #5014

Merged

niksaveliev mentioned this pull request May 30, 2024

Forbid autoscaling with size retention #5054

Merged

StekPerepolnen mentioned this pull request May 31, 2024

vdisk evict #5093

Merged

This was referenced Jun 6, 2024

Mute split by load ut #5264

Closed

Mute split by load ut #5267

Closed

Mute split by load ut #5272

Merged

Fix build #5310

Merged

serbel324 mentioned this pull request Jun 7, 2024

Don't mark slow disks if there is only one #5340

Merged

niksaveliev mentioned this pull request Jun 10, 2024

Fix split by load ut #5383

Merged

This was referenced Jun 11, 2024

metadata cache keeps merged issues #5461

Merged

json_hc metadata cache #5465

Merged

json tenantinfo status metadata cache #5543

Merged

json query transaction mode #5574

Merged

niksaveliev mentioned this pull request Jun 17, 2024

Fix win build #5622

Closed

This was referenced Jun 19, 2024

hc api doc #5703

Merged

move time difference issue under database level #5859

Merged

metadata requests flags fix #5860

Merged

StekPerepolnen mentioned this pull request Jun 28, 2024

oidc-proxy new iam 2 #6059

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

YQL-18052: Introduce CLI option to purebench to toggle block engine mode #4526

YQL-18052: Introduce CLI option to purebench to toggle block engine mode #4526

igormunkin commented May 14, 2024 •

edited

Loading

github-actions bot commented May 14, 2024 •

edited

Loading

github-actions bot commented May 14, 2024 •

edited

Loading

github-actions bot commented May 14, 2024 •

edited

Loading

github-actions bot commented May 15, 2024 •

edited

Loading

github-actions bot commented May 15, 2024 •

edited

Loading

github-actions bot commented May 15, 2024 •

edited

Loading

github-actions bot commented May 17, 2024 •

edited

Loading

github-actions bot commented May 17, 2024 •

edited

Loading

github-actions bot commented May 17, 2024 •

edited

Loading

github-actions bot commented May 28, 2024 •

edited

Loading

github-actions bot commented May 28, 2024 •

edited

Loading

github-actions bot commented May 28, 2024 •

edited

Loading

YQL-18052: Introduce CLI option to purebench to toggle block engine mode #4526

YQL-18052: Introduce CLI option to purebench to toggle block engine mode #4526

Conversation

igormunkin commented May 14, 2024 • edited Loading

Changelog category

github-actions bot commented May 14, 2024 • edited Loading

github-actions bot commented May 14, 2024 • edited Loading

github-actions bot commented May 14, 2024 • edited Loading

github-actions bot commented May 15, 2024 • edited Loading

github-actions bot commented May 15, 2024 • edited Loading

github-actions bot commented May 15, 2024 • edited Loading

github-actions bot commented May 17, 2024 • edited Loading

github-actions bot commented May 17, 2024 • edited Loading

github-actions bot commented May 17, 2024 • edited Loading

github-actions bot commented May 28, 2024 • edited Loading

github-actions bot commented May 28, 2024 • edited Loading

github-actions bot commented May 28, 2024 • edited Loading

igormunkin commented May 14, 2024 •

edited

Loading

github-actions bot commented May 14, 2024 •

edited

Loading

github-actions bot commented May 14, 2024 •

edited

Loading

github-actions bot commented May 14, 2024 •

edited

Loading

github-actions bot commented May 15, 2024 •

edited

Loading

github-actions bot commented May 15, 2024 •

edited

Loading

github-actions bot commented May 15, 2024 •

edited

Loading

github-actions bot commented May 17, 2024 •

edited

Loading

github-actions bot commented May 17, 2024 •

edited

Loading

github-actions bot commented May 17, 2024 •

edited

Loading

github-actions bot commented May 28, 2024 •

edited

Loading

github-actions bot commented May 28, 2024 •

edited

Loading

github-actions bot commented May 28, 2024 •

edited

Loading