Ydb configure features, part 2 #13920

Jorres · 2025-01-28T11:10:22Z

Changelog entry

...

Changelog category

Not for changelog (changelog entry is not required)

log_config

Now the config.yaml format of log_config is also supported in template.yaml:

log_config:
  default_level: 5
  format: json

The above is equivalent to the previous template.yaml:

log:
  default: 5
  format: json

There is no reason to use an old template.yaml format of this field.

domains_config

Now the config.yaml format of domains_config is also supported in template.yaml:

domains_config:
  domain:
  - name: ydb-slice-1
    storage_pool_types:
    - kind: ssd
      pool_config:
        box_id: 1
        erasure_species: mirror-3-dc
        kind: ssd
        geometry:
          realm_level_begin: 10
          realm_level_end: 20
          domain_level_begin: 10
          domain_level_end: 256
        pdisk_filter:
        - property:
          - type: SSD
        vdisk_kind: Default
  state_storage:
  - ring:
      node: [1, 2, 3]
      nto_select: 3
    ssid: 1

The old format looked something like this:

domains:
  - domain_name: ydb-slice-1
    storage_pool_kinds:
      - kind: "ssd"
        erasure: mirror-3-dc
        fail_domain_type: disk
        filter_properties:
          type: SSD
    console_initializers:
      - cloud_ssd_table_profile

There is no reason to use an old template.yaml format of this field.

cms_config

Now you can specify it directly in template.yaml, for example:

cms_config:
  sentinel_config:
    default_state_limit: 1

This would be validated and dumped into config.yaml, as with all the other fields

Minor fixes

forbid_implicit_storage_pools: True is removed from ydb_configure codebase as it is the default in YDB
ydb_configure no longer produces both ssl_port and port both equal to the same value in grpc_config

github-actions · 2025-01-28T11:14:21Z

⚪ 2025-01-28 11:14:20 UTC Pre-commit check linux-x86_64-release-asan for bbfaf09 has started.
⚪ 2025-01-28 11:14:32 UTC Artifacts will be uploaded here
⚪ 2025-01-28 11:17:08 UTC ya make is running...
🟡 2025-01-28 11:40:41 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
1643	1609	0	6	15	13

⚪ 2025-01-28 11:40:56 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-01-28 11:52:02 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
40 (only retried tests)	21	0	3	8	8

⚪ 2025-01-28 11:52:15 UTC ya make is running... (failed tests rerun, try 3)
🟢 2025-01-28 11:55:20 UTC ydbd size 3.6 GiB changed* by -96 Bytes, which is <= 0 Bytes vs main: OK

ydbd size dash	main: `777ff9a`	merge: `bbfaf09`	diff	diff %
ydbd size	3 864 980 680 Bytes	3 864 980 584 Bytes	-96 Bytes	-0.000%
ydbd stripped size	1 351 619 888 Bytes	1 351 619 824 Bytes	-64 Bytes	-0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}
⚫ 2025-01-28 11:55:22 UTC Check cancelled

github-actions · 2025-01-28T11:14:34Z

⚪ 2025-01-28 11:14:33 UTC Pre-commit check linux-x86_64-relwithdebinfo for bbfaf09 has started.
⚪ 2025-01-28 11:14:45 UTC Artifacts will be uploaded here
⚪ 2025-01-28 11:17:23 UTC ya make is running...
🟢 2025-01-28 11:41:23 UTC Tests successful.

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
4746	4620	0	0	37	89

🟢 2025-01-28 11:41:51 UTC Build successful.
🟢 2025-01-28 11:42:07 UTC ydbd size 2.1 GiB changed* by 0 Bytes, which is <= 0 Bytes vs main: OK

ydbd size dash	main: `777ff9a`	merge: `bbfaf09`	diff	diff %
ydbd size	2 224 267 728 Bytes	2 224 267 728 Bytes	0 Bytes	0.000%
ydbd stripped size	470 252 016 Bytes	470 252 016 Bytes	0 Bytes	0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2025-01-28T11:57:25Z

⚪ 2025-01-28 11:57:24 UTC Pre-commit check linux-x86_64-release-asan for f075aeb has started.
⚪ 2025-01-28 11:57:55 UTC Artifacts will be uploaded here
⚪ 2025-01-28 12:01:05 UTC ya make is running...
🟡 2025-01-28 12:27:55 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
1643	1605	0	7	16	15

⚪ 2025-01-28 12:28:11 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-01-28 12:39:27 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
44 (only retried tests)	25	0	2	8	9

⚪ 2025-01-28 12:39:37 UTC ya make is running... (failed tests rerun, try 3)
🟡 2025-01-28 12:50:34 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
24 (only retried tests)	12	0	2	2	8

🟢 2025-01-28 12:50:46 UTC Build successful.
🟢 2025-01-28 12:51:02 UTC ydbd size 3.6 GiB changed* by -6.2 KiB, which is <= 0 Bytes vs main: OK

ydbd size dash	main: `777ff9a`	merge: `f075aeb`	diff	diff %
ydbd size	3 864 980 680 Bytes	3 864 974 376 Bytes	-6.2 KiB	-0.000%
ydbd stripped size	1 351 619 888 Bytes	1 351 618 032 Bytes	-1.8 KiB	-0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2025-01-28T11:58:35Z

⚪ 2025-01-28 11:58:35 UTC Pre-commit check linux-x86_64-relwithdebinfo for f075aeb has started.
⚪ 2025-01-28 11:58:47 UTC Artifacts will be uploaded here
⚪ 2025-01-28 12:01:26 UTC ya make is running...
🟡 2025-01-28 12:39:19 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
4746	4609	0	2	40	95

⚪ 2025-01-28 12:39:56 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-01-28 12:48:18 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
110 (only retried tests)	30	0	0	0	80

🟢 2025-01-28 12:48:26 UTC Build successful.
🟢 2025-01-28 12:48:39 UTC ydbd size 2.1 GiB changed* by -3.2 KiB, which is <= 0 Bytes vs main: OK

ydbd size dash	main: `a89d4c9`	merge: `f075aeb`	diff	diff %
ydbd size	2 224 267 728 Bytes	2 224 264 464 Bytes	-3.2 KiB	-0.000%
ydbd stripped size	470 252 016 Bytes	470 251 248 Bytes	-768 Bytes	-0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2025-01-30T16:26:44Z

⚪ 2025-01-30 16:26:43 UTC Pre-commit check linux-x86_64-relwithdebinfo for 9fcec55 has started.
⚪ 2025-01-30 16:26:49 UTC Artifacts will be uploaded here
⚪ 2025-01-30 16:29:17 UTC ya make is running...
🟡 2025-01-30 17:39:16 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
4752	4620	0	10	38	84

⚪ 2025-01-30 17:40:10 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-01-30 17:48:29 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
112 (only retried tests)	32	0	0	0	80

🟢 2025-01-30 17:48:40 UTC Build successful.
🟢 2025-01-30 17:48:52 UTC ydbd size 2.1 GiB changed* by +11.8 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `f6f62ce`	merge: `9fcec55`	diff	diff %
ydbd size	2 226 979 896 Bytes	2 226 991 976 Bytes	+11.8 KiB	+0.001%
ydbd stripped size	471 309 840 Bytes	471 312 464 Bytes	+2.6 KiB	+0.001%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2025-01-30T16:28:26Z

⚪ 2025-01-30 16:28:25 UTC Pre-commit check linux-x86_64-release-asan for 9fcec55 has started.
⚪ 2025-01-30 16:28:31 UTC Artifacts will be uploaded here
⚪ 2025-01-30 16:31:06 UTC ya make is running...
🟡 2025-01-30 16:54:05 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
1649	1611	0	9	15	14

⚪ 2025-01-30 16:54:36 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-01-30 17:06:16 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
47 (only retried tests)	34	0	1	4	8

⚪ 2025-01-30 17:06:31 UTC ya make is running... (failed tests rerun, try 3)
🟢 2025-01-30 17:18:06 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
16 (only retried tests)	5	0	0	2	9

🟢 2025-01-30 17:18:19 UTC Build successful.
🟢 2025-01-30 17:18:36 UTC ydbd size 3.6 GiB changed* by -96 Bytes, which is <= 0 Bytes vs main: OK

ydbd size dash	main: `9db5077`	merge: `9fcec55`	diff	diff %
ydbd size	3 869 569 800 Bytes	3 869 569 704 Bytes	-96 Bytes	-0.000%
ydbd stripped size	1 353 426 000 Bytes	1 353 425 936 Bytes	-64 Bytes	-0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

Jorres added 5 commits January 28, 2025 11:30

feat: support log_config

7e96848

fix: grpc_config port vs ssl_port

c79f046

feat: support domains_config

da489bc

fix: some unused code cleanup

0a9b22b

fix: lint

25a32a2

github-actions bot added the not-for-changelog label Jan 28, 2025

chore: remove forbid_implicit_storage_pools as it is the default

de09262

github-actions bot added not-for-changelog and removed not-for-changelog labels Jan 28, 2025

feat: support cms_config in config.yaml style

299b9de

github-actions bot added not-for-changelog and removed not-for-changelog labels Jan 28, 2025

mvgorbunov previously approved these changes Jan 30, 2025

View reviewed changes

wip

192f957

Jorres dismissed mvgorbunov’s stale review via 192f957 January 30, 2025 16:22

mvgorbunov approved these changes Jan 30, 2025

View reviewed changes

Jorres enabled auto-merge (squash) January 30, 2025 16:24

Jorres merged commit bb6255c into ydb-platform:main Jan 30, 2025
12 checks passed

azevaykin pushed a commit to azevaykin/ydb that referenced this pull request Feb 3, 2025

Ydb configure features, part 2 (ydb-platform#13920)

4085ea6

GrigoriyPA mentioned this pull request Feb 11, 2025

YQ-4103 fixed overflow in kqp write actor CalculateNextAttemptDelay #14415

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Ydb configure features, part 2 #13920

Ydb configure features, part 2 #13920

Jorres commented Jan 28, 2025 •

edited

Loading

github-actions bot commented Jan 28, 2025 •

edited

Loading

github-actions bot commented Jan 28, 2025 •

edited

Loading

github-actions bot commented Jan 28, 2025 •

edited

Loading

github-actions bot commented Jan 28, 2025 •

edited

Loading

github-actions bot commented Jan 30, 2025 •

edited

Loading

github-actions bot commented Jan 30, 2025 •

edited

Loading

Ydb configure features, part 2 #13920

Ydb configure features, part 2 #13920

Conversation

Jorres commented Jan 28, 2025 • edited Loading

Changelog entry

Changelog category

log_config

domains_config

cms_config

Minor fixes

github-actions bot commented Jan 28, 2025 • edited Loading

github-actions bot commented Jan 28, 2025 • edited Loading

github-actions bot commented Jan 28, 2025 • edited Loading

github-actions bot commented Jan 28, 2025 • edited Loading

github-actions bot commented Jan 30, 2025 • edited Loading

github-actions bot commented Jan 30, 2025 • edited Loading

Jorres commented Jan 28, 2025 •

edited

Loading

github-actions bot commented Jan 28, 2025 •

edited

Loading

github-actions bot commented Jan 28, 2025 •

edited

Loading

github-actions bot commented Jan 28, 2025 •

edited

Loading

github-actions bot commented Jan 28, 2025 •

edited

Loading

github-actions bot commented Jan 30, 2025 •

edited

Loading

github-actions bot commented Jan 30, 2025 •

edited

Loading