BTreeIndex Bugfix partially-loaded tree #1674

kunga · 2024-02-07T13:31:39Z

Changelog entry

...

Changelog category

Not for changelog (changelog entry is not required)

Additional information

Previous version has a bug, that if we have some (but not all) pages, it passes wrong bounds to groups/history precharge.

For example:

3-leveled tree
- level0: node00: rows [0..40)
- level1: node10: rows [0..20); node11: rows [20..40)
- level2: node20: rows [0..10); node20: rows [10..20); ...
in memory we have all the nodes except node11
precharge keys range [5..15]

We will reach and stop on level 1, and have beginRowId = 0. And will call DoGroupsAndHistory(0, 10) instead of DoGroupsAndHistory(5, 10).

So instead of stopping as soon as we have unloaded node on current level, we should continue the process and search for exact begin/end bounds.

Also, it was really hard to track limits in previous version because we don't have enough information about nodes sizes and items (and skip unneeded nodes easily).

We may also consider 'Deep First Search' instead of current 'Breadth First Search' and do not store all level node states, but I expect that usually level will consist of two or three nodes.

github-actions · 2024-02-07T13:46:24Z

⚪ 2024-02-07 13:46:23 UTC Pre-commit check for 42b3845 has started.
⚪ 2024-02-07 13:46:25 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-02-07 13:48:02 UTC Build successful.
⚪ 2024-02-07 13:48:11 UTC Tests are running...
🔴 2024-02-07 15:05:03 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
67229	56365	0	11	10820	33

github-actions · 2024-02-07T13:52:47Z

⚪ 2024-02-07 13:52:46 UTC Pre-commit check for 42b3845 has started.
⚪ 2024-02-07 13:52:49 UTC Build linux-x86_64-release-asan is running...
🟢 2024-02-07 13:54:32 UTC Build successful.
⚪ 2024-02-07 13:54:42 UTC Tests are running...
🔴 2024-02-07 15:29:42 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
14574	14455	0	29	66	24

github-actions · 2024-02-07T16:00:52Z

⚪ 2024-02-07 16:00:51 UTC Pre-commit check for c8108fc has started.
⚪ 2024-02-07 16:00:54 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-02-07 16:02:58 UTC Build successful.
⚪ 2024-02-07 16:03:12 UTC Tests are running...
🔴 2024-02-07 16:43:21 UTC Test run completed, no test results found for commit c3b175b. Please check build logs.
⚫ 2024-02-07 16:43:25 UTC Check cancelled

github-actions · 2024-02-07T16:00:58Z

⚪ 2024-02-07 16:00:57 UTC Pre-commit check for c8108fc has started.
⚪ 2024-02-07 16:01:00 UTC Build linux-x86_64-release-asan is running...
🟢 2024-02-07 16:02:46 UTC Build successful.
⚪ 2024-02-07 16:03:00 UTC Tests are running...
🔴 2024-02-07 16:43:24 UTC Test run completed, no test results found for commit c3b175b. Please check build logs.
⚫ 2024-02-07 16:43:27 UTC Check cancelled

github-actions · 2024-02-07T16:44:03Z

⚪ 2024-02-07 16:44:03 UTC Pre-commit check for f2273fe has started.
⚪ 2024-02-07 16:44:05 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-02-07 16:45:42 UTC Build successful.
⚪ 2024-02-07 16:45:51 UTC Tests are running...
🔴 2024-02-07 18:08:19 UTC Test run completed, no test results found for commit 66441e7. Please check build logs.
⚫ 2024-02-07 18:08:23 UTC Check cancelled

github-actions · 2024-02-07T16:44:44Z

⚪ 2024-02-07 16:44:44 UTC Pre-commit check for f2273fe has started.
⚪ 2024-02-07 16:44:46 UTC Build linux-x86_64-release-asan is running...
🟢 2024-02-07 16:46:25 UTC Build successful.
⚪ 2024-02-07 16:46:35 UTC Tests are running...
🔴 2024-02-07 18:08:18 UTC Test run completed, no test results found for commit 66441e7. Please check build logs.
⚫ 2024-02-07 18:08:22 UTC Check cancelled

github-actions · 2024-02-07T18:10:17Z

⚪ 2024-02-07 18:10:16 UTC Pre-commit check for 3bd9d4f has started.
⚪ 2024-02-07 18:10:19 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-02-07 18:12:03 UTC Build successful.
⚪ 2024-02-07 18:12:16 UTC Tests are running...
🔴 2024-02-07 19:34:12 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
67247	56376	0	27	10820	24

github-actions · 2024-02-07T18:10:21Z

⚪ 2024-02-07 18:10:21 UTC Pre-commit check for 3bd9d4f has started.
⚪ 2024-02-07 18:10:23 UTC Build linux-x86_64-release-asan is running...
🟢 2024-02-07 18:12:23 UTC Build successful.
⚪ 2024-02-07 18:12:35 UTC Tests are running...
🔴 2024-02-07 19:48:52 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
14594	14441	0	57	56	40

github-actions · 2024-02-08T13:24:03Z

⚪ 2024-02-08 13:24:03 UTC Pre-commit check for ed31bb4 has started.
⚪ 2024-02-08 13:24:05 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-02-08 13:25:59 UTC Build successful.
⚪ 2024-02-08 13:26:12 UTC Tests are running...
🔴 2024-02-08 13:58:26 UTC Test run completed, no test results found for commit 8cefe19. Please check build logs.
⚫ 2024-02-08 13:58:30 UTC Check cancelled

github-actions · 2024-02-08T13:24:11Z

⚪ 2024-02-08 13:24:10 UTC Pre-commit check for ed31bb4 has started.
⚪ 2024-02-08 13:24:13 UTC Build linux-x86_64-release-asan is running...
🟢 2024-02-08 13:26:03 UTC Build successful.
⚪ 2024-02-08 13:26:15 UTC Tests are running...
🔴 2024-02-08 13:58:21 UTC Test run completed, no test results found for commit 8cefe19. Please check build logs.
⚫ 2024-02-08 13:58:24 UTC Check cancelled

github-actions · 2024-02-08T13:59:17Z

⚪ 2024-02-08 13:59:16 UTC Pre-commit check for 5cf4969 has started.
⚪ 2024-02-08 13:59:18 UTC Build linux-x86_64-release-asan is running...
🟢 2024-02-08 14:13:07 UTC Build successful.
⚪ 2024-02-08 14:13:16 UTC Tests are running...
🔴 2024-02-08 15:53:33 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
14738	14547	0	18	132	41

github-actions · 2024-02-08T13:59:56Z

⚪ 2024-02-08 13:59:55 UTC Pre-commit check for 5cf4969 has started.
⚪ 2024-02-08 13:59:57 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-02-08 14:13:08 UTC Build successful.
⚪ 2024-02-08 14:13:18 UTC Tests are running...
🟢 2024-02-08 15:52:01 UTC Tests successful.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
67413	56461	0	0	10892	60

kunga added 2 commits February 6, 2024 18:21

do not stop groups charge if isn't ready

591c2d2

stick pages

cebb3ad

github-actions bot added the not-for-changelog label Feb 7, 2024

kunga changed the title ~~BTreeIndex Bugfix curvy tree~~ BTreeIndex Bugfix partially-loaded tree Feb 7, 2024

child state

c3b175b

fix bytes limit

66441e7

clean up

cc512ec

kunga added 3 commits February 7, 2024 20:46

DoGroupsAndHistory

5d22dec

refactor reverse

39172bb

fix check

8cefe19

kunga mentioned this pull request Feb 8, 2024

👑 Readonly B-Tree SST index #1483

Open

61 tasks

remove unused

8d51432

kunga requested a review from snaury February 8, 2024 14:14

kunga marked this pull request as ready for review February 8, 2024 14:14

snaury approved these changes Feb 9, 2024

View reviewed changes

kunga merged commit bb72994 into ydb-platform:main Feb 9, 2024
3 of 4 checks passed

pavelvelikhov mentioned this pull request Feb 9, 2024

Cannonized two plans #1773

Merged

shnikd mentioned this pull request Feb 12, 2024

Add default policy to ColumnMeta #1791

Merged

niksaveliev mentioned this pull request Feb 12, 2024

Workload read without consumer and metrics fixes #1792

Merged

starlinskiy mentioned this pull request Feb 12, 2024

runtime dispatching #1800

Closed

This was referenced Feb 12, 2024

quick fix for column name of select version() #1801

Merged

Support of database sys cache #1817

Merged

starlinskiy mentioned this pull request Feb 12, 2024

runtime dispatching (fixed) #1847

Merged

serbel324 mentioned this pull request Feb 13, 2024

Improve some headers #1877

Merged

pavelvelikhov mentioned this pull request Feb 13, 2024

Fixed a problem with simplified plan JSONs #1878

Merged

serbel324 mentioned this pull request Feb 13, 2024

Reintroduce test_helpers for ut_blobstorage #1890

Merged

This was referenced Feb 13, 2024

Case insensivity for pg_catalog tables/cluster, columns. Support of pg_class:relam, and some columns in pg_database & pg_namespace #1893

Merged

fixed pg_get_userbyid function #1936

Merged

support of AnyNonArray type, fixes for array_agg #1958

Merged

pavelvelikhov mentioned this pull request Feb 15, 2024

Fixed a problem with plans in PG ut #1974

Merged

niksaveliev mentioned this pull request Feb 16, 2024

Fix kafka with enabled proxy #2017

Merged

vitstn mentioned this pull request Feb 16, 2024

mock of savepoints #2026

Merged

GrigoriyPA mentioned this pull request Feb 17, 2024

YQ 2786 unmute s3 test bindings #2042

Merged

shnikd mentioned this pull request Feb 18, 2024

Add from_sequence option to default value policy #2050

Merged

This was referenced Feb 19, 2024

YQ-2884 fixed fq cancel operation race #2055

Merged

YQ-2883 fix retries for recover point COMPLETING #2107

Merged

YQ-2873 fix hanging in kqp compute actor #2131

Merged

YQ-2786 partly unmute tests fq s3 #2143

Merged

GrigoriyPA mentioned this pull request Feb 26, 2024

YQ-2898 fix query failed with unknown uncommitted upload issue #2240

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

BTreeIndex Bugfix partially-loaded tree #1674

BTreeIndex Bugfix partially-loaded tree #1674

kunga commented Feb 7, 2024 •

edited

Loading

github-actions bot commented Feb 7, 2024 •

edited

Loading

github-actions bot commented Feb 7, 2024 •

edited

Loading

github-actions bot commented Feb 7, 2024 •

edited

Loading

github-actions bot commented Feb 7, 2024 •

edited

Loading

github-actions bot commented Feb 7, 2024 •

edited

Loading

github-actions bot commented Feb 7, 2024 •

edited

Loading

github-actions bot commented Feb 7, 2024 •

edited

Loading

github-actions bot commented Feb 7, 2024 •

edited

Loading

github-actions bot commented Feb 8, 2024 •

edited

Loading

github-actions bot commented Feb 8, 2024 •

edited

Loading

github-actions bot commented Feb 8, 2024 •

edited

Loading

github-actions bot commented Feb 8, 2024 •

edited

Loading

BTreeIndex Bugfix partially-loaded tree #1674

BTreeIndex Bugfix partially-loaded tree #1674

Conversation

kunga commented Feb 7, 2024 • edited Loading

Changelog entry

Changelog category

Additional information

github-actions bot commented Feb 7, 2024 • edited Loading

github-actions bot commented Feb 7, 2024 • edited Loading

github-actions bot commented Feb 7, 2024 • edited Loading

github-actions bot commented Feb 7, 2024 • edited Loading

github-actions bot commented Feb 7, 2024 • edited Loading

github-actions bot commented Feb 7, 2024 • edited Loading

github-actions bot commented Feb 7, 2024 • edited Loading

github-actions bot commented Feb 7, 2024 • edited Loading

github-actions bot commented Feb 8, 2024 • edited Loading

github-actions bot commented Feb 8, 2024 • edited Loading

github-actions bot commented Feb 8, 2024 • edited Loading

github-actions bot commented Feb 8, 2024 • edited Loading

kunga commented Feb 7, 2024 •

edited

Loading

github-actions bot commented Feb 7, 2024 •

edited

Loading

github-actions bot commented Feb 7, 2024 •

edited

Loading

github-actions bot commented Feb 7, 2024 •

edited

Loading

github-actions bot commented Feb 7, 2024 •

edited

Loading

github-actions bot commented Feb 7, 2024 •

edited

Loading

github-actions bot commented Feb 7, 2024 •

edited

Loading

github-actions bot commented Feb 7, 2024 •

edited

Loading

github-actions bot commented Feb 7, 2024 •

edited

Loading

github-actions bot commented Feb 8, 2024 •

edited

Loading

github-actions bot commented Feb 8, 2024 •

edited

Loading

github-actions bot commented Feb 8, 2024 •

edited

Loading

github-actions bot commented Feb 8, 2024 •

edited

Loading