Refactoring to make cardinality estimation provider-specific #3866

alephonea · 2024-04-18T09:52:30Z

Changelog entry

...

Additional information

...

github-actions · 2024-04-18T14:39:28Z

⚪ 2024-04-18 14:39:27 UTC Pre-commit check for f0fc9b9 has started.
⚪ 2024-04-18 14:39:29 UTC Build linux-x86_64-release-clang14 is running...
🔴 2024-04-18 14:45:16 UTC Build failed. see the build logs.

github-actions · 2024-04-18T14:47:22Z

⚪ 2024-04-18 14:47:21 UTC Pre-commit check for f0fc9b9 has started.
⚪ 2024-04-18 14:47:23 UTC Build linux-x86_64-relwithdebinfo is running...
🔴 2024-04-18 14:52:04 UTC Build failed. see the build logs.
🔴 2024-04-18 14:53:30 UTC Tests run skipped.

github-actions · 2024-04-18T15:33:46Z

⚪ 2024-04-18 15:33:45 UTC Pre-commit check for f0fc9b9 has started.
⚪ 2024-04-18 15:33:48 UTC Build linux-x86_64-release-asan is running...
🔴 2024-04-18 15:39:05 UTC Build failed. see the build logs.
🔴 2024-04-18 15:40:41 UTC Tests run skipped.

github-actions · 2024-04-18T16:37:20Z

⚪ 2024-04-18 16:37:19 UTC Pre-commit check for d7a843e has started.
⚪ 2024-04-18 16:37:21 UTC Build linux-x86_64-release-clang14 is running...
🔴 2024-04-18 16:52:18 UTC Build failed. see the build logs.

github-actions · 2024-04-18T16:39:58Z

⚪ 2024-04-18 16:39:58 UTC Pre-commit check for d7a843e has started.
⚪ 2024-04-18 16:40:01 UTC Build linux-x86_64-relwithdebinfo is running...
🔴 2024-04-18 16:55:37 UTC Build failed. see the build logs.
🔴 2024-04-18 16:57:10 UTC Tests run skipped.

github-actions · 2024-04-18T16:44:21Z

⚪ 2024-04-18 16:44:20 UTC Pre-commit check for d7a843e has started.
⚪ 2024-04-18 16:44:22 UTC Build linux-x86_64-release-asan is running...
🔴 2024-04-18 17:03:17 UTC Build failed. see the build logs.
🔴 2024-04-18 17:04:55 UTC Tests run skipped.

pavelvelikhov · 2024-04-18T17:32:23Z

ydb/library/yql/core/yql_statistics.h

@@ -28,6 +34,7 @@ struct TOptimizerStatistics {
    double Cost = 0;
    double Selectivity = 1.0;
    const TVector<TString>& KeyColumns;
+    const IProviderStatistics* Specific = nullptr;


This will work okay only if you own all Specific objects outside the optimizer and release the memory outside.
But this means you can't use Specific for intermediate optimizer nodes, only for base relations

github-actions · 2024-04-19T12:22:59Z

⚪ 2024-04-19 12:22:59 UTC Pre-commit check for 005c0f1 has started.
⚪ 2024-04-19 12:23:00 UTC Build linux-x86_64-release-clang14 is running...
🔴 2024-04-19 12:36:22 UTC Build failed. see the build logs.

github-actions · 2024-04-19T12:43:34Z

⚪ 2024-04-19 12:43:34 UTC Pre-commit check for 005c0f1 has started.
⚪ 2024-04-19 12:43:37 UTC Build linux-x86_64-relwithdebinfo is running...
🔴 2024-04-19 12:56:54 UTC Build failed. see the build logs.
🔴 2024-04-19 12:58:29 UTC Tests run skipped.

github-actions · 2024-04-19T13:05:24Z

⚪ 2024-04-19 13:05:24 UTC Pre-commit check for 005c0f1 has started.
⚪ 2024-04-19 13:05:26 UTC Build linux-x86_64-release-asan is running...
🔴 2024-04-19 13:19:14 UTC Build failed. see the build logs.
🔴 2024-04-19 13:20:50 UTC Tests run skipped.

github-actions · 2024-04-19T13:27:21Z

⚪ 2024-04-19 13:27:21 UTC Pre-commit check for c9e17d9 has started.
⚪ 2024-04-19 13:27:23 UTC Build linux-x86_64-release-asan is running...
🔴 2024-04-19 13:45:58 UTC Build failed. see the build logs.
🔴 2024-04-19 13:47:38 UTC Tests run skipped.

github-actions · 2024-04-19T13:27:39Z

⚪ 2024-04-19 13:27:39 UTC Pre-commit check for c9e17d9 has started.
⚪ 2024-04-19 13:27:40 UTC Build linux-x86_64-relwithdebinfo is running...
🔴 2024-04-19 13:45:32 UTC Build failed. see the build logs.
🔴 2024-04-19 13:47:17 UTC Tests run skipped.

github-actions · 2024-04-19T13:29:20Z

⚪ 2024-04-19 13:29:19 UTC Pre-commit check for c9e17d9 has started.
⚪ 2024-04-19 13:29:22 UTC Build linux-x86_64-release-clang14 is running...
🔴 2024-04-19 13:44:39 UTC Build failed. see the build logs.

pavelvelikhov

We also need to fix our KqpProviderContext to subclass from TDummyProviderContext (renamed to something less insulting)

pavelvelikhov · 2024-04-19T15:09:31Z

ydb/library/yql/core/cbo/cbo_optimizer_new.cpp

+    return true;
+}
+
+bool TDummyProviderContext::IsJoinApplicable(const std::shared_ptr<IBaseOptimizerNode>& left,


Lets rename TDummyProviderContext to something like TBaseProviderContext or TDefaultProviderContext.

Its not that dummy anymore if we're including the standard cardinality estimation in there

github-actions · 2024-04-19T16:09:58Z

⚪ 2024-04-19 16:09:58 UTC Pre-commit check for e8a492a has started.
⚪ 2024-04-19 16:10:01 UTC Build linux-x86_64-relwithdebinfo is running...
🔴 2024-04-19 16:13:40 UTC Build failed. see the build logs.
🔴 2024-04-19 16:15:14 UTC Tests run skipped.

github-actions · 2024-04-19T16:10:01Z

⚪ 2024-04-19 16:10:01 UTC Pre-commit check for e8a492a has started.
⚪ 2024-04-19 16:10:03 UTC Build linux-x86_64-release-clang14 is running...
🔴 2024-04-19 16:13:48 UTC Build failed. see the build logs.

github-actions · 2024-04-19T16:10:04Z

⚪ 2024-04-19 16:10:04 UTC Pre-commit check for e8a492a has started.
⚪ 2024-04-19 16:10:07 UTC Build linux-x86_64-release-asan is running...
🔴 2024-04-19 16:13:37 UTC Build failed. see the build logs.
🔴 2024-04-19 16:15:15 UTC Tests run skipped.

github-actions · 2024-04-19T16:36:12Z

⚪ 2024-04-19 16:36:11 UTC Pre-commit check for ce64e4c has started.
⚪ 2024-04-19 16:36:13 UTC Build linux-x86_64-release-asan is running...
🔴 2024-04-19 16:39:35 UTC Build failed. see the build logs.
🔴 2024-04-19 16:41:10 UTC Tests run skipped.

github-actions · 2024-04-19T16:36:16Z

⚪ 2024-04-19 16:36:15 UTC Pre-commit check for ce64e4c has started.
⚪ 2024-04-19 16:36:18 UTC Build linux-x86_64-release-clang14 is running...
🔴 2024-04-19 16:40:06 UTC Build failed. see the build logs.

github-actions · 2024-04-19T16:36:21Z

⚪ 2024-04-19 16:36:20 UTC Pre-commit check for ce64e4c has started.
⚪ 2024-04-19 16:36:23 UTC Build linux-x86_64-relwithdebinfo is running...
🔴 2024-04-19 16:40:02 UTC Build failed. see the build logs.
🔴 2024-04-19 16:41:36 UTC Tests run skipped.

github-actions · 2024-04-22T17:00:24Z

⚪ 2024-04-22 17:00:23 UTC Pre-commit check for a9abcd7 has started.
⚪ 2024-04-22 17:00:25 UTC Build linux-x86_64-release-asan is running...
🔴 2024-04-22 17:17:49 UTC Build failed. see the build logs.
🔴 2024-04-22 17:19:15 UTC Tests run skipped.

github-actions · 2024-04-22T17:01:21Z

⚪ 2024-04-22 17:01:20 UTC Pre-commit check for a9abcd7 has started.
⚪ 2024-04-22 17:01:23 UTC Build linux-x86_64-relwithdebinfo is running...
🔴 2024-04-22 17:18:01 UTC Build failed. see the build logs.
🔴 2024-04-22 17:19:40 UTC Tests run skipped.

github-actions · 2024-04-22T17:01:28Z

⚪ 2024-04-22 17:01:27 UTC Pre-commit check for a9abcd7 has started.
⚪ 2024-04-22 17:01:30 UTC Build linux-x86_64-release-clang14 is running...
🔴 2024-04-22 17:15:59 UTC Build failed. see the build logs.

alephonea · 2024-05-13T22:20:23Z

We also need to fix our KqpProviderContext to subclass from TDummyProviderContext (renamed to something less insulting)

alephonea requested review from a team as code owners April 18, 2024 09:52

github-actions bot added the new-feature label Apr 18, 2024

alephonea force-pushed the cbo-exp branch from ed9b62a to 9b4f90d Compare April 18, 2024 14:14

pavelvelikhov requested changes Apr 18, 2024

View reviewed changes

alephonea force-pushed the cbo-exp branch from 04f6eeb to 27ad45d Compare April 19, 2024 12:11

pavelvelikhov requested changes Apr 19, 2024

View reviewed changes

Refactoring to perform cardinality estimation specifically for YT.

e7a9e84

alephonea force-pushed the cbo-exp branch from 22d2ca8 to e7a9e84 Compare April 22, 2024 16:59

pavelvelikhov approved these changes Apr 23, 2024

View reviewed changes

alephonea closed this May 13, 2024

alephonea mentioned this pull request May 13, 2024

Refactoring to perform cardinality estimation specifically for YT. #4482

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactoring to make cardinality estimation provider-specific #3866

Refactoring to make cardinality estimation provider-specific #3866

alephonea commented Apr 18, 2024

github-actions bot commented Apr 18, 2024 •

edited

Loading

github-actions bot commented Apr 18, 2024 •

edited

Loading

github-actions bot commented Apr 18, 2024 •

edited

Loading

github-actions bot commented Apr 18, 2024 •

edited

Loading

github-actions bot commented Apr 18, 2024 •

edited

Loading

github-actions bot commented Apr 18, 2024 •

edited

Loading

pavelvelikhov Apr 18, 2024

github-actions bot commented Apr 19, 2024 •

edited

Loading

github-actions bot commented Apr 19, 2024 •

edited

Loading

github-actions bot commented Apr 19, 2024 •

edited

Loading

github-actions bot commented Apr 19, 2024 •

edited

Loading

github-actions bot commented Apr 19, 2024 •

edited

Loading

github-actions bot commented Apr 19, 2024 •

edited

Loading

pavelvelikhov left a comment

pavelvelikhov Apr 19, 2024

alephonea May 13, 2024

github-actions bot commented Apr 19, 2024 •

edited

Loading

github-actions bot commented Apr 19, 2024 •

edited

Loading

github-actions bot commented Apr 19, 2024 •

edited

Loading

github-actions bot commented Apr 19, 2024 •

edited

Loading

github-actions bot commented Apr 19, 2024 •

edited

Loading

github-actions bot commented Apr 19, 2024 •

edited

Loading

github-actions bot commented Apr 22, 2024 •

edited

Loading

github-actions bot commented Apr 22, 2024 •

edited

Loading

github-actions bot commented Apr 22, 2024 •

edited

Loading

alephonea commented May 13, 2024

Refactoring to make cardinality estimation provider-specific #3866

Refactoring to make cardinality estimation provider-specific #3866

Conversation

alephonea commented Apr 18, 2024

Changelog entry

Changelog category

Additional information

github-actions bot commented Apr 18, 2024 • edited Loading

github-actions bot commented Apr 18, 2024 • edited Loading

github-actions bot commented Apr 18, 2024 • edited Loading

github-actions bot commented Apr 18, 2024 • edited Loading

github-actions bot commented Apr 18, 2024 • edited Loading

github-actions bot commented Apr 18, 2024 • edited Loading

pavelvelikhov Apr 18, 2024

Choose a reason for hiding this comment

github-actions bot commented Apr 19, 2024 • edited Loading

github-actions bot commented Apr 19, 2024 • edited Loading

github-actions bot commented Apr 19, 2024 • edited Loading

github-actions bot commented Apr 19, 2024 • edited Loading

github-actions bot commented Apr 19, 2024 • edited Loading

github-actions bot commented Apr 19, 2024 • edited Loading

pavelvelikhov left a comment

Choose a reason for hiding this comment

pavelvelikhov Apr 19, 2024

Choose a reason for hiding this comment

alephonea May 13, 2024

Choose a reason for hiding this comment

github-actions bot commented Apr 19, 2024 • edited Loading

github-actions bot commented Apr 19, 2024 • edited Loading

github-actions bot commented Apr 19, 2024 • edited Loading

github-actions bot commented Apr 19, 2024 • edited Loading

github-actions bot commented Apr 19, 2024 • edited Loading

github-actions bot commented Apr 19, 2024 • edited Loading

github-actions bot commented Apr 22, 2024 • edited Loading

github-actions bot commented Apr 22, 2024 • edited Loading

github-actions bot commented Apr 22, 2024 • edited Loading

alephonea commented May 13, 2024

github-actions bot commented Apr 18, 2024 •

edited

Loading

github-actions bot commented Apr 18, 2024 •

edited

Loading

github-actions bot commented Apr 18, 2024 •

edited

Loading

github-actions bot commented Apr 18, 2024 •

edited

Loading

github-actions bot commented Apr 18, 2024 •

edited

Loading

github-actions bot commented Apr 18, 2024 •

edited

Loading

github-actions bot commented Apr 19, 2024 •

edited

Loading

github-actions bot commented Apr 19, 2024 •

edited

Loading

github-actions bot commented Apr 19, 2024 •

edited

Loading

github-actions bot commented Apr 19, 2024 •

edited

Loading

github-actions bot commented Apr 19, 2024 •

edited

Loading

github-actions bot commented Apr 19, 2024 •

edited

Loading

github-actions bot commented Apr 19, 2024 •

edited

Loading

github-actions bot commented Apr 19, 2024 •

edited

Loading

github-actions bot commented Apr 19, 2024 •

edited

Loading

github-actions bot commented Apr 19, 2024 •

edited

Loading

github-actions bot commented Apr 19, 2024 •

edited

Loading

github-actions bot commented Apr 19, 2024 •

edited

Loading

github-actions bot commented Apr 22, 2024 •

edited

Loading

github-actions bot commented Apr 22, 2024 •

edited

Loading

github-actions bot commented Apr 22, 2024 •

edited

Loading