Disable merges for indexImplTables partitions when build is in progress

[jepett0]

Ticket:

• index restoration: apply min partition count implicitly when restoring from backup #8583

Description of the change

Do not merge partitions of indexImplTables if they are not finished being built (i.e. the state of the index object is not equal to EIndexStateReady).

Context

We have implemented saving split boundaries of indexImplTables to the backup recently:

• Restore indexes from backup with the original partitioning #7589

However, the indexImplTable is created empty during the process of an import from S3. The data is being added to it at the filling stage of the BuildIndexes phase of the import process. If there are a lot of partitions in the index, it would take a lot of time to fill them. During the filling stage, SchemeShard might decide to merge the partitions of this indexImplTable. We would like to prevent it, because it will slow down the import process significantly. DataShards cannot build index if its partitions are being merged and would have to wait for the merge to end.

Notes

Commits are separated for your convince:

• preliminary refactoring: c92a1d7
• the main change: eb8cb41
• unit tests: ccb3ff4

[jepett0]

Please, evaluate the following design choice.

I have decided to use a structure defined in the public API to describe a TBuildIndexConfig in SchemeShard's unit tests.

Pros:

• We don't have to repeat the same code in the unit test helpers.

Cons:

• Using public API for unit test helpers might limit us greatly. We would have to change the public description of an index every time we decide to test some new functionality in the SchemeShard tests.
  • Counterargument: ut helpers already use a protobuf from the public API to describe an index to be created. It is only natural to use the SDK to work conveniently with the public protobuf. The dependency on the public API and the limitations that come with it would not be lifted, if we didn't use SDK here.

[github-actions]

⚪ 2024-10-10 15:38:20 UTC Pre-commit check linux-x86_64-relwithdebinfo for 7d9b343 has started.
⚪ 2024-10-10 15:38:31 UTC Artifacts will be uploaded here
⚪ 2024-10-10 15:41:21 UTC ya make is running...
🟡 2024-10-10 16:29:20 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED?
15294	13780	0	7	1385	122

⚪ 2024-10-10 16:30:38 UTC ya make is running... (failed tests rerun, try 2)
🟡 2024-10-10 16:41:56 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED?
131 (only retried tests)	27	0	1	0	103

⚪ 2024-10-10 16:42:05 UTC ya make is running... (failed tests rerun, try 3)
🟢 2024-10-10 16:53:30 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED?
104 (only retried tests)	5	0	0	0	99

🟢 2024-10-10 16:53:37 UTC Build successful.
🟢 2024-10-10 16:54:06 UTC ydbd size 2.8 GiB changed* by +1.5 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: 576bc68	merge: 7d9b343	diff	diff %
ydbd size	3 012 101 352 Bytes	3 012 102 888 Bytes	+1.5 KiB	+0.000%
ydbd stripped size	477 607 224 Bytes	477 607 096 Bytes	-128 Bytes	-0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

[github-actions]

⚪ 2024-10-10 15:39:51 UTC Pre-commit check linux-x86_64-release-asan for 7d9b343 has started.
⚪ 2024-10-10 15:40:02 UTC Artifacts will be uploaded here
⚪ 2024-10-10 15:42:54 UTC ya make is running...
🟡 2024-10-10 17:11:09 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED?
10354	10083	0	80	78	113

🟢 2024-10-10 17:12:02 UTC Build successful.
🟢 2024-10-10 17:12:38 UTC ydbd size 5.7 GiB changed* by +4.5 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: cb4e8c5	merge: 7d9b343	diff	diff %
ydbd size	6 103 994 696 Bytes	6 103 999 304 Bytes	+4.5 KiB	+0.000%
ydbd stripped size	1 521 013 872 Bytes	1 521 014 448 Bytes	+576 Bytes	+0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}