Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Filter pushdown optimization. #947

Merged
merged 17 commits into from
Dec 23, 2019
Merged

Conversation

CPWstatic
Copy link
Contributor

@CPWstatic CPWstatic commented Sep 17, 2019

The filter pushdown optimization is based on rewriting filter. Purpose of this pr is about to enhance the performance.

@CPWstatic CPWstatic added do not review PR: not ready for the code review yet ready-for-testing PR: ready for the CI test labels Sep 17, 2019
@nebula-community-bot
Copy link
Member

Unit testing passed.

@CPWstatic CPWstatic removed the do not review PR: not ready for the code review yet label Sep 18, 2019
@nebula-community-bot
Copy link
Member

Unit testing passed.

Copy link
Contributor

@critical27 critical27 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clean code

@nebula-community-bot
Copy link
Member

Unit testing passed.

Copy link
Contributor

@laura-ding laura-ding left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well done

@nebula-community-bot
Copy link
Member

Unit testing passed.

laura-ding
laura-ding previously approved these changes Sep 26, 2019
@nebula-community-bot
Copy link
Member

Unit testing passed.

1 similar comment
@nebula-community-bot
Copy link
Member

Unit testing passed.

@nebula-community-bot
Copy link
Member

Unit testing failed.

1 similar comment
@nebula-community-bot
Copy link
Member

Unit testing failed.

@nebula-community-bot
Copy link
Member

Unit testing passed.

1 similar comment
@nebula-community-bot
Copy link
Member

Unit testing passed.

std::function<OptVariantType()> getEdgeRank;
std::function<OptVariantType(const std::string&)> getInputProp;
std::function<OptVariantType(const std::string&)> getVariableProp;
std::function<OptVariantType(const std::string&, const std::string&)> getSrcTagProp;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we add some other interfaces like:

std::function<OptVariantType(TagID id, const std::string&)> getSrcTagProp
std::function<OptVariantType(EdgeType type, const std::string&)> getEdgeProp;


Copy link
Contributor

@dangleptr dangleptr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be honest, your rewrite logic looks some bit weird.
Typically, the recursive program should looks like this:

Expression*  rewrite(Expression*  expr) {
switch (expr->type()) {
case kLogic:
         auto* left = rewrite(expr->left());
         auto* write = rewrite(expr->right());
         return new LogicExpr(left,  right);
case ...



}

}

wdyt?

@nebula-community-bot
Copy link
Member

Unit testing passed.

@nebula-community-bot
Copy link
Member

Unit testing passed.

@nebula-community-bot
Copy link
Member

Unit testing passed.

dangleptr
dangleptr previously approved these changes Dec 20, 2019
Copy link
Contributor

@dangleptr dangleptr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome work!

@nebula-community-bot
Copy link
Member

Unit testing passed.

return buf;
}

OptVariantType AliasPropertyExpression::eval() const {
return context_->getters().getAliasProp(*alias_, *prop_);
OptVariantType AliasPropertyExpression::eval(Getters &getters) const {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

using const

@nebula-community-bot
Copy link
Member

Unit testing passed.

@dutor dutor merged commit c5621c9 into vesoft-inc:master Dec 23, 2019
@jude-zhu
Copy link
Contributor

relate to #175

tong-hao pushed a commit to tong-hao/nebula that referenced this pull request Jun 1, 2021
* Filter push down.

* Rewrite and.

* Add test.

* Rewrite xor,or.

* Not rewrite xor and add tests.

* Add test.

* Rebase and fix conflict.

* Fix encode/decode for expression.

* Make getters thread safe.

* Fix get reserved prop in key.

* Rebase and fix conflict.

* Add gflags to tell if pushdown filter.

* Address @dangleptr's comment.

* Push down functions.

* Test push down filter or not.

* Add ut for testing push down function.

Co-authored-by: dutor <440396+dutor@users.noreply.github.com>
yixinglu pushed a commit to yixinglu/nebula that referenced this pull request Jan 31, 2023
Co-authored-by: Doodle <13706157+critical27@users.noreply.github.com>

Co-authored-by: panda-sheep <59197347+panda-sheep@users.noreply.github.com>
Co-authored-by: Doodle <13706157+critical27@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready-for-testing PR: ready for the CI test
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants