s3 listing strategy has been fixed #9499

dorooleg · 2024-09-19T11:32:09Z

Changelog entry

...

Changelog category

Bugfix

Additional information

...

github-actions · 2024-09-19T11:35:34Z

⚪ 2024-09-19 11:35:33 UTC Pre-commit check linux-x86_64-release-asan for f01a435 has started.
⚪ 2024-09-19 11:35:43 UTC Artifacts will be uploaded here
⚪ 2024-09-19 11:38:33 UTC ya make is running...
🔴 2024-09-19 13:20:22 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
10307	10125	0	24	46	112

🟢 2024-09-19 13:21:15 UTC Build successful.
🟡 2024-09-19 13:21:49 UTC ydbd size 5.6 GiB changed* by +192.7 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash	main: `a7cb65e`	merge: `f01a435`	diff	diff %
ydbd size	6 058 347 352 Bytes	6 058 544 656 Bytes	+192.7 KiB	+0.003%
ydbd stripped size	1 515 536 240 Bytes	1 515 583 440 Bytes	+46.1 KiB	+0.003%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-09-19T11:35:34Z

⚪ 2024-09-19 11:35:34 UTC Pre-commit check linux-x86_64-release-clang14 for f01a435 has started.
⚪ 2024-09-19 11:35:43 UTC Artifacts will be uploaded here
⚪ 2024-09-19 11:38:33 UTC ya make is running...
🟢 2024-09-19 11:47:58 UTC Build successful.

github-actions · 2024-09-19T11:35:54Z

⚪ 2024-09-19 11:35:54 UTC Pre-commit check linux-x86_64-relwithdebinfo for f01a435 has started.
⚪ 2024-09-19 11:36:04 UTC Artifacts will be uploaded here
⚪ 2024-09-19 11:39:00 UTC ya make is running...
🟡 2024-09-19 12:49:13 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
39482	33914	0	18	5446	104

⚪ 2024-09-19 12:52:35 UTC ya make is running... (failed tests rerun, try 2)
🟡 2024-09-19 13:04:09 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
130 (only retried tests)	38	0	1	0	91

⚪ 2024-09-19 13:04:18 UTC ya make is running... (failed tests rerun, try 3)
🔴 2024-09-19 13:15:56 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
98 (only retried tests)	6	0	1	0	91

🟢 2024-09-19 13:16:03 UTC Build successful.
🟡 2024-09-19 13:16:44 UTC ydbd size 8.4 GiB changed* by +384.9 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash	main: `a7cb65e`	merge: `f01a435`	diff	diff %
ydbd size	9 040 535 752 Bytes	9 040 929 912 Bytes	+384.9 KiB	+0.004%
ydbd stripped size	489 103 112 Bytes	489 116 552 Bytes	+13.1 KiB	+0.003%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-09-19T19:26:44Z

⚪ 2024-09-19 19:26:43 UTC Pre-commit check linux-x86_64-release-clang14 for 9f16582 has started.
⚪ 2024-09-19 19:26:54 UTC Artifacts will be uploaded here
⚪ 2024-09-19 19:29:52 UTC ya make is running...
🟢 2024-09-19 19:36:16 UTC Build successful.

github-actions · 2024-09-19T19:27:10Z

⚪ 2024-09-19 19:27:10 UTC Pre-commit check linux-x86_64-release-asan for 9f16582 has started.
⚪ 2024-09-19 19:27:20 UTC Artifacts will be uploaded here
⚪ 2024-09-19 19:30:19 UTC ya make is running...
🔴 2024-09-19 21:21:35 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
10329	10099	0	62	56	112

🟢 2024-09-19 21:22:30 UTC Build successful.
🟢 2024-09-19 21:23:08 UTC ydbd size 5.6 GiB changed* by +16.7 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `ec8172a`	merge: `9f16582`	diff	diff %
ydbd size	6 060 458 936 Bytes	6 060 476 056 Bytes	+16.7 KiB	+0.000%
ydbd stripped size	1 516 088 176 Bytes	1 516 091 568 Bytes	+3.3 KiB	+0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-09-19T19:29:08Z

⚪ 2024-09-19 19:29:07 UTC Pre-commit check linux-x86_64-relwithdebinfo for 9f16582 has started.
⚪ 2024-09-19 19:29:17 UTC Artifacts will be uploaded here
⚪ 2024-09-19 19:32:06 UTC ya make is running...
🟡 2024-09-19 20:35:23 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
39486	33929	0	3	5449	105

⚪ 2024-09-19 20:38:39 UTC ya make is running... (failed tests rerun, try 2)
🟢 2024-09-19 20:49:56 UTC Tests successful.

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
122 (only retried tests)	27	0	0	0	95

🟢 2024-09-19 20:50:02 UTC Build successful.
🟢 2024-09-19 20:50:42 UTC ydbd size 8.4 GiB changed* by +9.2 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `ec8172a`	merge: `9f16582`	diff	diff %
ydbd size	9 041 890 960 Bytes	9 041 900 368 Bytes	+9.2 KiB	+0.000%
ydbd stripped size	489 265 288 Bytes	489 265 800 Bytes	+512 Bytes	+0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

uzhastik · 2024-09-20T07:59:29Z

ydb/library/yql/providers/s3/provider/yql_s3_listing_strategy.cpp

@@ -360,7 +365,7 @@ class TPartitionedDatasetS3ListingStrategy : public TCollectingS3ListingStrategy
                        result.ListedObjectSize = listingResult.ListedObjectSize;
                        for (auto& directoryPrefix : listingResult.Directories) {
                            if (directoryPrefix.MatchedGlobs.empty()) {
-                                // We need to list until extra columns are extracted
+                                // We need to list until extra columns are extracteds


Внес опечатку

github-actions · 2024-09-20T10:33:01Z

⚪ 2024-09-20 10:33:01 UTC Pre-commit check linux-x86_64-release-clang14 for a4c7cc1 has started.
⚪ 2024-09-20 10:33:10 UTC Artifacts will be uploaded here
⚪ 2024-09-20 10:36:03 UTC ya make is running...
🟢 2024-09-20 10:42:00 UTC Build successful.

github-actions · 2024-09-20T10:35:24Z

⚪ 2024-09-20 10:35:24 UTC Pre-commit check linux-x86_64-release-asan for a4c7cc1 has started.
⚪ 2024-09-20 10:35:33 UTC Artifacts will be uploaded here
⚪ 2024-09-20 10:38:33 UTC ya make is running...
🔴 2024-09-20 12:25:36 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
10322	10128	0	40	40	114

🟢 2024-09-20 12:26:35 UTC Build successful.
🟢 2024-09-20 12:27:10 UTC ydbd size 5.6 GiB changed* by -4.6 KiB, which is <= 0 Bytes vs main: OK

ydbd size dash	main: `77a5f3a`	merge: `a4c7cc1`	diff	diff %
ydbd size	6 060 680 096 Bytes	6 060 675 368 Bytes	-4.6 KiB	-0.000%
ydbd stripped size	1 516 142 384 Bytes	1 516 140 720 Bytes	-1.6 KiB	-0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2024-09-20T10:38:42Z

⚪ 2024-09-20 10:38:41 UTC Pre-commit check linux-x86_64-relwithdebinfo for a4c7cc1 has started.
⚪ 2024-09-20 10:38:52 UTC Artifacts will be uploaded here
⚪ 2024-09-20 10:41:44 UTC ya make is running...
🟡 2024-09-20 11:45:32 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
39489	33732	0	8	5712	37

⚪ 2024-09-20 11:48:56 UTC ya make is running... (failed tests rerun, try 2)
🟢 2024-09-20 12:00:07 UTC Tests successful.

Test history | Ya make output

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
325 (only retried tests)	225	0	0	8	92

🟢 2024-09-20 12:00:16 UTC Build successful.
🟢 2024-09-20 12:01:03 UTC ydbd size 8.4 GiB changed* by -12.2 KiB, which is <= 0 Bytes vs main: OK

ydbd size dash	main: `77a5f3a`	merge: `a4c7cc1`	diff	diff %
ydbd size	9 042 177 512 Bytes	9 042 165 056 Bytes	-12.2 KiB	-0.000%
ydbd stripped size	489 288 072 Bytes	489 287 240 Bytes	-832 Bytes	-0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

Co-authored-by: Oleg Doronin <hcpp@rtmr-dev01.search.yandex.net>

s3 listing strategy has been fixed

732f892

dorooleg requested a review from a team as a code owner September 19, 2024 11:32

github-actions bot added the bugfix label Sep 19, 2024

flake has been fixed

5d3b4e5

uzhastik reviewed Sep 20, 2024

View reviewed changes

misspelling has been fixed

246207e

uzhastik approved these changes Sep 25, 2024

View reviewed changes

dorooleg merged commit 041ec57 into ydb-platform:main Sep 25, 2024
10 of 12 checks passed

dorooleg deleted the fix_s3_listing_strategy_v2 branch September 25, 2024 15:19

dorooleg added a commit to dorooleg/ydb that referenced this pull request Sep 26, 2024

s3 listing strategy has been fixed (ydb-platform#9499)

baf459c

dorooleg added a commit to dorooleg/ydb that referenced this pull request Sep 26, 2024

s3 listing strategy has been fixed (ydb-platform#9499)

9e5a6f6

dorooleg added a commit to dorooleg/ydb that referenced this pull request Sep 26, 2024

s3 listing strategy has been fixed (ydb-platform#9499)

d9ca689

dorooleg added a commit that referenced this pull request Sep 27, 2024

merge from main: s3 listing strategy has been fixed (#9499) (#9821)

97e9a76

Co-authored-by: Oleg Doronin <hcpp@rtmr-dev01.search.yandex.net>

This was referenced Sep 29, 2024

YQ-3704 RD use CH udf in json parsing #9878

Merged

YQ-3703 pushdown for extract members in PQ provider #9939

Merged

GrigoriyPA mentioned this pull request Oct 8, 2024

YQ-3722 add simdjson parser into RD #10204

Merged

This was referenced Oct 21, 2024

Fix kafka cdc read and login without @ #10678

Merged

Allow short auto partitioning strategies yql #10701

Merged

GrigoriyPA mentioned this pull request Oct 29, 2024

YQ RD pass UV from parser to filter #11078

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

s3 listing strategy has been fixed #9499

s3 listing strategy has been fixed #9499

dorooleg commented Sep 19, 2024

github-actions bot commented Sep 19, 2024 •

edited

Loading

github-actions bot commented Sep 19, 2024 •

edited

Loading

github-actions bot commented Sep 19, 2024 •

edited

Loading

github-actions bot commented Sep 19, 2024 •

edited

Loading

github-actions bot commented Sep 19, 2024 •

edited

Loading

github-actions bot commented Sep 19, 2024 •

edited

Loading

uzhastik Sep 20, 2024

github-actions bot commented Sep 20, 2024 •

edited

Loading

github-actions bot commented Sep 20, 2024 •

edited

Loading

github-actions bot commented Sep 20, 2024 •

edited

Loading

s3 listing strategy has been fixed #9499

s3 listing strategy has been fixed #9499

Conversation

dorooleg commented Sep 19, 2024

Changelog entry

Changelog category

Additional information

github-actions bot commented Sep 19, 2024 • edited Loading

github-actions bot commented Sep 19, 2024 • edited Loading

github-actions bot commented Sep 19, 2024 • edited Loading

github-actions bot commented Sep 19, 2024 • edited Loading

github-actions bot commented Sep 19, 2024 • edited Loading

github-actions bot commented Sep 19, 2024 • edited Loading

uzhastik Sep 20, 2024

Choose a reason for hiding this comment

github-actions bot commented Sep 20, 2024 • edited Loading

github-actions bot commented Sep 20, 2024 • edited Loading

github-actions bot commented Sep 20, 2024 • edited Loading

github-actions bot commented Sep 19, 2024 •

edited

Loading

github-actions bot commented Sep 19, 2024 •

edited

Loading

github-actions bot commented Sep 19, 2024 •

edited

Loading

github-actions bot commented Sep 19, 2024 •

edited

Loading

github-actions bot commented Sep 19, 2024 •

edited

Loading

github-actions bot commented Sep 19, 2024 •

edited

Loading

github-actions bot commented Sep 20, 2024 •

edited

Loading

github-actions bot commented Sep 20, 2024 •

edited

Loading

github-actions bot commented Sep 20, 2024 •

edited

Loading