Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix optimizer symbols. #4106

Conversation

Shylock-Hg
Copy link
Contributor

@Shylock-Hg Shylock-Hg commented Mar 31, 2022

What type of PR is this?

  • bug
  • feature
  • enhancement

What problem(s) does this PR solve?

Issue(s) number:

Close #3826

Description:

Add two constraints to optimizer:

  1. One OptGroup will share same output variable
  2. There is no overlap output variable between OptGroup

And add one implicit transform:

  1. The transformed result of optimizer rule will rebuild input relationship by dependencies.

And some improvement:

  1. PlanNode clone will generate new output variable and use it
  2. PlanNode only need one output variable so change it from vector to scalar.

How do you solve it?

Special notes for your reviewer, ex. impact of this fix, design document, etc:

Checklist:

Tests:

  • Unit test(positive and negative cases)
  • Function test
  • Performance test
  • N/A

Affects:

  • Documentation affected (Please add the label if documentation needs to be modified.)
  • Incompatibility (If it breaks the compatibility, please describe it and add the label.)
  • If it's needed to cherry-pick (If cherry-pick to some branches is required, please label the destination version(s).)
  • Performance impacted: Consumes more CPU/Memory

Release notes:

Please confirm whether to be reflected in release notes and how to describe:

ex. Fixed the bug .....

@Shylock-Hg Shylock-Hg added ready-for-testing PR: ready for the CI test type/bug Type: something is unexpected labels Mar 31, 2022
Copy link
Contributor

@yixinglu yixinglu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Compared to help the OptRule author to rebuild the dependency relationships, I prefer to check them in optimizer and report a big warning for them if they don't do the right thing. This will help author know where the error is.

src/graph/optimizer/OptGroup.cpp Show resolved Hide resolved
for (std::size_t i = 0; i < dependencies_.size(); i++) {
const OptGroup *dep = dependencies_[i];
node_->setInputVar(dep->outputVar(), i);
for (auto *groupNode : dep->groupNodes()) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need not to rebuild all OptGroupNodes in the depended OptGroup since there are many unchanged nodes which are not in the path of the OptRule. It will save much time to rebuild the nodes of transformed by the OptRule. WDYT?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could let users do it in rule, and only check it here. As you said before.

@@ -31,6 +31,13 @@ const PlanNode *MatchedResult::planNode(const std::vector<int32_t> &pos) const {
return DCHECK_NOTNULL(result->node)->node();
}

void MatchedResult::collectBoundary(std::vector<OptGroup *> &boundary) const {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's better to collect all OptGroupNode rather than OptGroup in the path of matched result.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But the dependencies_ is OptGroup.

@@ -30,6 +30,8 @@ class OptGroup;
struct MatchedResult {
const OptGroupNode *node{nullptr};
std::vector<MatchedResult> dependencies;
// Boundary of the whole matched result, aka the dependencies of leaf node in MatchedResult.
std::vector<OptGroup *> boundary_;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like there is no need to save this variable.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we could collect it in match method directly.

@Shylock-Hg
Copy link
Contributor Author

Replaced by #4146

@Shylock-Hg Shylock-Hg closed this Apr 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready-for-testing PR: ready for the CI test type/bug Type: something is unexpected
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Optimizer now only release symbols of top layer nodes in all matched nodes.
2 participants