BTreeIndex Iterator Remove single run optimization #995

kunga · 2024-01-15T10:27:55Z

Seek benchmarks yag test -r -F benchmark::TPartIndexIteratorFixture/DoReads/*/0/0/*/*/1

(TPartIndexIteratorFixture/DoReads/b-tree/groups=0/history=0/reverse/ESeek/items=1)

Before:

----------------------------------------------------------------------------------------
Benchmark                                              Time             CPU   Iterations
----------------------------------------------------------------------------------------
TPartIndexIteratorFixture/DoReads/0/0/0/0/1/1       18.8 us         7.74 us        89449
TPartIndexIteratorFixture/DoReads/1/0/0/0/1/1       7.41 us         7.39 us        93294
TPartIndexIteratorFixture/DoReads/0/0/0/1/1/1       7.42 us         7.21 us        98974
TPartIndexIteratorFixture/DoReads/1/0/0/1/1/1       7.58 us         7.57 us        93598
TPartIndexIteratorFixture/DoReads/0/0/0/0/2/1       8.22 us         6.94 us       104775
TPartIndexIteratorFixture/DoReads/1/0/0/0/2/1       8.43 us         7.26 us        96510
TPartIndexIteratorFixture/DoReads/0/0/0/1/2/1       12.5 us         7.20 us       105019
TPartIndexIteratorFixture/DoReads/1/0/0/1/2/1       7.14 us         7.13 us        96782
TPartIndexIteratorFixture/DoReads/0/0/0/0/3/1       13.6 us         7.58 us        88728
TPartIndexIteratorFixture/DoReads/1/0/0/0/3/1       9.47 us         7.64 us        93685
TPartIndexIteratorFixture/DoReads/0/0/0/1/3/1       9.68 us         6.96 us       102325
TPartIndexIteratorFixture/DoReads/1/0/0/1/3/1       11.0 us         7.05 us        97635

After:

----------------------------------------------------------------------------------------
Benchmark                                              Time             CPU   Iterations
----------------------------------------------------------------------------------------
TPartIndexIteratorFixture/DoReads/0/0/0/0/1/1       11.4 us         7.37 us        94450
TPartIndexIteratorFixture/DoReads/1/0/0/0/1/1       9.49 us         7.77 us        92168
TPartIndexIteratorFixture/DoReads/0/0/0/1/1/1       7.87 us         7.18 us        99113
TPartIndexIteratorFixture/DoReads/1/0/0/1/1/1       9.71 us         7.69 us        86517
TPartIndexIteratorFixture/DoReads/0/0/0/0/2/1       10.2 us         7.37 us        90860
TPartIndexIteratorFixture/DoReads/1/0/0/0/2/1       10.3 us         7.61 us        93749
TPartIndexIteratorFixture/DoReads/0/0/0/1/2/1       7.81 us         6.87 us        96138
TPartIndexIteratorFixture/DoReads/1/0/0/1/2/1       8.95 us         7.36 us        99316
TPartIndexIteratorFixture/DoReads/0/0/0/0/3/1       9.64 us         7.18 us        96818
TPartIndexIteratorFixture/DoReads/1/0/0/0/3/1       11.0 us         7.77 us        95898
TPartIndexIteratorFixture/DoReads/0/0/0/1/3/1       15.1 us         7.36 us       103496
TPartIndexIteratorFixture/DoReads/1/0/0/1/3/1       7.41 us         7.37 us        94385

No differences noticed

github-actions · 2024-01-15T12:24:38Z

Note

This is an automated comment that will be appended during run.

🔴 linux-x86_64-relwithdebinfo: some tests FAILED for commit 0208b85.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
59902	50681	0	1	9207	13

🔴 linux-x86_64-release-asan: some tests FAILED for commit 0208b85.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
15887	15751	0	23	103	10

snaury · 2024-01-17T08:52:33Z

ydb/core/tablet_flat/test/libs/table/test_part.h

@@ -176,16 +176,15 @@ namespace NTest {
            return index.GetLastRecord();
        }

-        inline const TPartIndexIt::TRecord * GetRecord(const TPartStore& part, TPageId pageId) {
+        inline const TPartIndexIt::TRecord * GetRecord(const TPartStore& part, TPageId pageIndex) {


Очень минорное: немного странное переименование pageId -> pageIndex, ведь индекс это и есть page id.

pageIndex это типо номер с начала внутри TPart

например TPart из 10 страниц их индексы [0..9]

а pageId могут быть [0, 1, 5, 10, 15, ..

В таком случае использовать TPageId в качестве типа очень странно.

kunga added 5 commits January 15, 2024 19:01

remove run.size=1 optimization

e08cb14

bench with seek modes

6a5b019

wip test with slices

7ea2625

cook slices

c6b1758

fix tests

0208b85

kunga force-pushed the KIKIMR-19521-iter-check-run-bounds branch from 7db9a00 to 0208b85 Compare January 15, 2024 19:02

kunga marked this pull request as ready for review January 15, 2024 19:03

kunga requested a review from snaury January 15, 2024 21:37

snaury approved these changes Jan 17, 2024

View reviewed changes

kunga merged commit 5fa4b2f into main Jan 17, 2024
3 of 5 checks passed

kunga deleted the KIKIMR-19521-iter-check-run-bounds branch January 22, 2024 12:46

pavelvelikhov mentioned this pull request Jan 24, 2024

Fixed a minor bug in simplified plan JSON generation #1257

Merged

FloatingCrowbar mentioned this pull request Jan 24, 2024

Fix args order bug #1280

Merged

niksaveliev mentioned this pull request Jan 25, 2024

Fix kafka metadata actor for Topics.size() == 0 and fix read by short topic name #1287

Merged

vitstn mentioned this pull request Jan 25, 2024

validate & lineage modes in dqrun #1290

Merged

aakulaga-ydb mentioned this pull request Jan 25, 2024

JsonDocument support added YQL-17658 #1306

Merged

This was referenced Jan 25, 2024

YQL-17418 Support of Merge in lineage #1307

Merged

Fix windows build for invoke_builtins/no_llvm #1315

Merged

YQL-17264 static part of tables #1327

Merged

YQL-17678 throw exception instead of abort #1348

Merged

YQL-17264 static part of columns #1354

Merged

niksaveliev mentioned this pull request Jan 29, 2024

Kafka Connect docs #1369

Merged

vitstn mentioned this pull request Jan 29, 2024

fix mkql warnings #1375

Merged

shnikd mentioned this pull request Jan 29, 2024

KIKIMR-20082: TableStore/ColumnTable with QueryService #1377

Merged

This was referenced Jan 29, 2024

fix warnings for pg_wrapper #1383

Merged

fix sql2yql warnings #1394

Merged

fix pg sql warnings #1395

Merged

pavelvelikhov mentioned this pull request Jan 29, 2024

Enabled multiple join implementations in CBO #1396

Merged

vitstn mentioned this pull request Jan 29, 2024

YQL-17343 information_schema.table_constraints #1397

Merged

shnikd mentioned this pull request Feb 5, 2024

Support temp tables in yql #1589

Merged

This was referenced Feb 5, 2024

YQL-17339 import all columns from real PG #1594

Merged

fix aarch64 compile flags #1598

Merged

niksaveliev mentioned this pull request Feb 5, 2024

Fixes lock partition delay and balance test #1600

Merged

pavelvelikhov mentioned this pull request Feb 5, 2024

Generate single JSON with all types of plans in the server #1606

Merged

vitstn mentioned this pull request Feb 5, 2024

YQL-17476 check for cycle during evaluation of file args #1608

Merged

This was referenced Feb 6, 2024

Fix iterator at kafka connect #1611

Closed

Fix iterator at kafka connect #1612

Merged

This was referenced Feb 6, 2024

parallel for #1650

Merged

YQL-17725 drop b and t suffixes #1660

Merged

YQL-17725 tune doc #1667

Merged

Switch pg_parser BC tests on again #1709

Merged

unmute test #1711

Merged

ildar-khisambeev mentioned this pull request Feb 8, 2024

LOGBROKER-8783 bugfix #1713

Merged

vitstn mentioned this pull request Feb 8, 2024

Mark mutable_id for YT writes #1744

Merged

niksaveliev mentioned this pull request Feb 9, 2024

Fix pq writer and few renames #1757

Merged

pavelvelikhov mentioned this pull request Feb 9, 2024

Cannonized two plans #1773

Merged

niksaveliev mentioned this pull request Feb 12, 2024

Workload read without consumer and metrics fixes #1792

Merged

starlinskiy mentioned this pull request Feb 12, 2024

runtime dispatching #1800

Closed

This was referenced Feb 12, 2024

quick fix for column name of select version() #1801

Merged

Support of database sys cache #1817

Merged

starlinskiy mentioned this pull request Feb 12, 2024

runtime dispatching (fixed) #1847

Merged

pavelvelikhov mentioned this pull request Feb 13, 2024

Fixed a problem with simplified plan JSONs #1878

Merged

This was referenced Feb 13, 2024

Case insensivity for pg_catalog tables/cluster, columns. Support of pg_class:relam, and some columns in pg_database & pg_namespace #1893

Merged

fixed pg_get_userbyid function #1936

Merged

support of AnyNonArray type, fixes for array_agg #1958

Merged

pavelvelikhov mentioned this pull request Feb 15, 2024

Fixed a problem with plans in PG ut #1974

Merged

niksaveliev mentioned this pull request Feb 16, 2024

Fix kafka with enabled proxy #2017

Merged

vitstn mentioned this pull request Feb 16, 2024

mock of savepoints #2026

Merged

aakulaga-ydb mentioned this pull request Feb 24, 2024

Fix TzTimestamp wrong encoding (YQL-17920) #2222

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

BTreeIndex Iterator Remove single run optimization #995

BTreeIndex Iterator Remove single run optimization #995

kunga commented Jan 15, 2024 •

edited

Loading

github-actions bot commented Jan 15, 2024 •

edited

Loading

snaury Jan 17, 2024 •

edited

Loading

kunga Jan 17, 2024 •

edited

Loading

snaury Jan 18, 2024

BTreeIndex Iterator Remove single run optimization #995

BTreeIndex Iterator Remove single run optimization #995

Conversation

kunga commented Jan 15, 2024 • edited Loading

github-actions bot commented Jan 15, 2024 • edited Loading

snaury Jan 17, 2024 • edited Loading

Choose a reason for hiding this comment

kunga Jan 17, 2024 • edited Loading

Choose a reason for hiding this comment

snaury Jan 18, 2024

Choose a reason for hiding this comment

kunga commented Jan 15, 2024 •

edited

Loading

github-actions bot commented Jan 15, 2024 •

edited

Loading

snaury Jan 17, 2024 •

edited

Loading

kunga Jan 17, 2024 •

edited

Loading