[Relay] Pass manager #2546

zhiics · 2019-02-02T05:38:46Z

We have opened a RFC (#2512) for the pass manager. This PR implements the framework of it. Currently, we support the following type of passes:

module-level passes, they mainly focuses on optimizations cross functions, like inter-procedural optimizations/analysis, addition and removal of functions.
function-level passes, they focuses on optimizations in a relay function scope

Each pass works on a module. Passes could be registered in python and sent to c++. An optimize function is provided to accept a host of passes for analysis and optimization. The current PR only executes these passes without tuning the ordering.

One example could be:

class MyClass():
    def __init__(self, mod):
        self.module = mod

    def transform(node, pass_ctx):
        """Perform optimization on node."""
        pass

my_class = MyClass(mod)
def transform(expr, pass_ctx):
    return my_class.transform(expr, pass_ctx)

module_pass = ModulePass("module_pass", 1, transform)
function_pass = FunctionPass("function_pass", 2, transform)

mod = relay.Module(...)
passes = [module_pass, function_pass]
sequential_pass = pass_manager.SequentialPass("sequential_pass", 2, passes)
ret_mod = sequential_pass(mod)

ret_mod contains all the updated information

CC' @jroesch @wweic @ZihengJiang @tqchen @junrushao1994, @icemelon9, @FrozenGene, @MarisaKirisame please review. Thanks.

tqchen · 2019-02-02T06:51:40Z

Let us spend a bit more time in discussing the Pass API as per https://docs.tvm.ai/contribute/code_review.html#deliberate-on-api-and-data-structures, I make some followup comments in the RFC

zhiics · 2019-02-02T07:29:43Z

@tqchen No problem, thanks. Let us discuss in the RFC.

include/tvm/relay/optimizer.h

src/relay/ir/module.cc

tests/python/relay/test_pass_optimizer.py

wweic

overall lgtm.

include/tvm/relay/optimizer.h

include/tvm/relay/pass_manager.h

python/tvm/relay/_pass_manager.pyi

python/tvm/relay/pass_manager.py

tests/python/relay/test_pass_manager.py

tqchen · 2019-03-01T02:42:36Z

@zhiics @jroesch I made some further comments, mainly on minimum interface design, please check

include/tvm/relay/pass_manager.h

zhiics · 2019-03-02T07:43:29Z

@tqchen fixed your comments, PTAL. Thanks.

zhiics · 2019-03-02T07:56:04Z

Not sure why quantized_pass failed. Seems it is not related to this PR.

include/tvm/relay/pass.h

python/tvm/relay/pass_manager.py

tqchen · 2019-03-02T18:26:38Z

Some additional pass

include/tvm/relay/pass.h

tqchen · 2019-03-03T01:12:11Z

include/tvm/relay/pass.h

+          typename = std::enable_if<(std::is_same<NodeT, Module>::value ||
+                                     std::is_same<NodeT, Function>::value)>>
+using PassFunc =
+    runtime::TypedPackedFunc<runtime::TypedPackedFunc<NodeT(NodeT)>(


We can do (NodeT, PassContext)->NodeT, so you do not capture the module, the outer pass will call this function to transform the corresponding module.

tqchen

minor nit. General interface lgtm, please tag more reviewers

include/tvm/relay/pass.h

zhiics · 2019-03-03T18:52:22Z

@wweic @jroesch @yzhliu @junrushao1994, @icemelon9 Please take a look when you have time. Thanks.

include/tvm/relay/pass.h

python/tvm/relay/ir_pass.py

src/relay/pass/pass_manager.cc

include/tvm/relay/pass.h

wweic

lgtm. two additional doc comment.

python/tvm/relay/ir_pass.py

tqchen

wait pass info change per #2512

zhiics · 2019-03-08T05:25:44Z

@tqchen @jroesch @FrozenGene updated. Let's merge this and move on if it looks good to you guys.

python/tvm/relay/ir_pass.py

src/relay/pass/pass_manager.cc

include/tvm/relay/pass.h

python/tvm/relay/ir_pass.py

src/relay/pass/pass_manager.cc

zhiics · 2019-03-10T18:58:41Z

@tqchen Updated. Please take another looks.

python/tvm/relay/ir_pass.py

tqchen · 2019-03-10T19:15:23Z

python/tvm/relay/ir_pass.py

+        The callback function that sketches a certain optimization.
+    """
+
+    def __init__(self, name, opt_level, pass_func, required=None):


I am not sure if we really want to support constructor along with the construction function here. Perhaps we can just not define constructor and ask the user to use create the function

I feel like it is probably necessary to leave it here so users have a flexibility to apply a pass_func directly without using decorators. I am okay with either way. I can remove it if you think it is unnecessary

tqchen · 2019-03-10T19:17:03Z

python/tvm/relay/ir_pass.py

+                                            disabled)
+
+
+def create_module_pass(pass_name, opt_level, pass_func, required=None):


We might want to consider enable a decorator pattern here.

# Allow automatic derivation of pass name from the function name @relay.ir_pass.module_pass(opt_level=2, required=[x, y, z]) def my_module_pass(mod, cx): ...

src/relay/pass/pass_manager.cc

tqchen · 2019-03-10T19:18:46Z

tests/python/relay/test_pass_quantize.py

-    res0 = graph.evaluate(qgraph0)(dataset[0]['data'])
-    res1 = graph.evaluate(qgraph1)(dataset[0]['data'])
-    tvm.testing.assert_allclose(res0.asnumpy(), res1.asnumpy(), rtol=1e-3)
+# def test_quantize_pass():


try to keep these as it is.

We talked to Ziheng and this test is broken, he needs to fix it, he said he would open follow-up PR.

Yeah, I think this pass has been also failing for me after I rebased it a while ago. @ZihengJiang said I can comment it out for now. He will look into it.

tqchen

See my additional comment on API design.

We should be really careful to make consistent argument orders between all APIs

tqchen · 2019-03-11T04:12:06Z

python/tvm/relay/ir_pass.py

+                                            disabled)
+
+
+def module_pass(opt_level, name=None, required=None):


Consistency in the convention:

Let us make the argument list as follows:

def module_pass(pass_func=None, opt_level=None, name=None, required=None): """When pass_func == None, decorator mod, otherwise normal mode. """

Please also change all the constructors to keep the consistent order of arguments

API design is really important if we take a look at constructor order and the function order they are different.
This can be quite problematic and confuses user in the long run :)

Thanks. Using this signature, now I think we should remove those initialization functions.

tqchen · 2019-03-11T04:13:12Z

python/tvm/relay/ir_pass.py

+    return create_function_pass
+
+
+def sequential_pass(opt_level, passes, name="sequential_pass", required=None,


To be consistent with other functions, argument order should be:

passes

name, can have a default value "sequential_pass"

opt_level can have a default value.

We can find that this order is in particular inconsistent with others.

python/tvm/relay/ir_pass.py

@tqchen

* initial commit * add python frontend and module tests * add unit tests for function pass and optimize interface * add ExprPass * remove PassState and pass context for run * add required_passes * return module * remove move * fix minor reviews * remove optimizer, optimizer->pass_manager, make pass a the base class of all * remove deleted files * move resolvedependency to sequential pass, use ir_pass namespace * add todo * add disabled passes in sequetialpass * fix minor * fix currying doc * remove pass_kind from passnode * remove pass kind from test * fix doc * fix per @tqchen's comments * remove pass_manager.py create separate classes * simplify pass_func * inline using passfunc * update doc * disable test_quantize_pass for now * create PassInfo class to contain the meta data * flatten passinfo for interface * retrigger ci * remove required method * make Pass python class lighter * create pass -> decorator * make the api consistent for all classes

@tqchen

* initial commit * add python frontend and module tests * add unit tests for function pass and optimize interface * add ExprPass * remove PassState and pass context for run * add required_passes * return module * remove move * fix minor reviews * remove optimizer, optimizer->pass_manager, make pass a the base class of all * remove deleted files * move resolvedependency to sequential pass, use ir_pass namespace * add todo * add disabled passes in sequetialpass * fix minor * fix currying doc * remove pass_kind from passnode * remove pass kind from test * fix doc * fix per @tqchen's comments * remove pass_manager.py create separate classes * simplify pass_func * inline using passfunc * update doc * disable test_quantize_pass for now * create PassInfo class to contain the meta data * flatten passinfo for interface * retrigger ci * remove required method * make Pass python class lighter * create pass -> decorator * make the api consistent for all classes

zhiics changed the title ~~Pass manager~~ [Relay] Pass manager Feb 2, 2019

jroesch reviewed Feb 14, 2019

View reviewed changes

include/tvm/relay/optimizer.h Outdated Show resolved Hide resolved

zhiics mentioned this pull request Feb 16, 2019

[Relay][RFC] Pass Manager #2512

Closed

wweic reviewed Feb 18, 2019

View reviewed changes

include/tvm/relay/optimizer.h Outdated Show resolved Hide resolved

src/relay/ir/module.cc Outdated Show resolved Hide resolved

tests/python/relay/test_pass_optimizer.py Outdated Show resolved Hide resolved

zhiics force-pushed the pass_manager branch from 4f6ff4c to 2f4e8ee Compare February 19, 2019 00:59

wweic reviewed Feb 19, 2019

View reviewed changes

include/tvm/relay/optimizer.h Outdated Show resolved Hide resolved

tqchen self-assigned this Feb 19, 2019

jroesch reviewed Feb 27, 2019

View reviewed changes

zhiics force-pushed the pass_manager branch from 12ff25a to 11040fc Compare February 27, 2019 07:06

tqchen requested changes Mar 1, 2019

View reviewed changes

include/tvm/relay/pass_manager.h Outdated Show resolved Hide resolved

include/tvm/relay/pass_manager.h Outdated Show resolved Hide resolved

zhiics force-pushed the pass_manager branch from 018c2fb to 498de9b Compare March 2, 2019 07:42

tqchen requested changes Mar 2, 2019

View reviewed changes

zhiics force-pushed the pass_manager branch from 061a490 to 5e0c802 Compare March 2, 2019 23:47

tqchen requested changes Mar 3, 2019

View reviewed changes

include/tvm/relay/pass.h Outdated Show resolved Hide resolved

wweic requested changes Mar 3, 2019

View reviewed changes

wweic approved these changes Mar 3, 2019

View reviewed changes

python/tvm/relay/ir_pass.py Outdated Show resolved Hide resolved

python/tvm/relay/ir_pass.py Outdated Show resolved Hide resolved

zhiics force-pushed the pass_manager branch from 940d51c to c386772 Compare March 3, 2019 22:55

wweic approved these changes Mar 3, 2019

View reviewed changes

zhiics force-pushed the pass_manager branch from c386772 to a722ecf Compare March 5, 2019 04:10

jroesch mentioned this pull request Mar 5, 2019

[TEST][FLAKY] test_quantize_pass #2718

Closed

zhiics force-pushed the pass_manager branch from c6df33a to 861e65c Compare March 5, 2019 20:40

jroesch approved these changes Mar 8, 2019

View reviewed changes

tqchen requested changes Mar 8, 2019

View reviewed changes

create PassInfo class to contain the meta data

49df272

tqchen requested changes Mar 8, 2019

View reviewed changes

python/tvm/relay/ir_pass.py Outdated Show resolved Hide resolved

MarisaKirisame reviewed Mar 8, 2019

View reviewed changes

src/relay/pass/pass_manager.cc Show resolved Hide resolved

zhiics added 2 commits March 8, 2019 19:39

flatten passinfo for interface

810cbac

retrigger ci

ee359fb

tqchen requested changes Mar 9, 2019

View reviewed changes

include/tvm/relay/pass.h Outdated Show resolved Hide resolved

remove required method

a0863b4

tqchen requested changes Mar 10, 2019

View reviewed changes

python/tvm/relay/ir_pass.py Outdated Show resolved Hide resolved

python/tvm/relay/ir_pass.py Outdated Show resolved Hide resolved

python/tvm/relay/ir_pass.py Show resolved Hide resolved

FrozenGene reviewed Mar 10, 2019

View reviewed changes

src/relay/pass/pass_manager.cc Show resolved Hide resolved

make Pass python class lighter

c8fb6b5

zhiics force-pushed the pass_manager branch from 30fafdd to c8fb6b5 Compare March 10, 2019 19:00

tqchen requested changes Mar 10, 2019

View reviewed changes

create pass -> decorator

6df5c7d

tqchen requested changes Mar 11, 2019

View reviewed changes

python/tvm/relay/ir_pass.py Show resolved Hide resolved

make the api consistent for all classes

e78f4e2

jroesch approved these changes Mar 11, 2019

View reviewed changes

tqchen approved these changes Mar 11, 2019

View reviewed changes

jroesch merged commit abe6f77 into apache:master Mar 12, 2019

zhiics deleted the pass_manager branch March 12, 2019 03:01

icemelon added the status: accepted label Mar 12, 2019

jroesch mentioned this pull request Mar 14, 2019

[RFC] Relay Dynamic Runtime #2810

Closed

tqchen mentioned this pull request May 17, 2019

[RFC][Relay] Pass API Discussion #3202

Closed

tqchen mentioned this pull request Nov 8, 2019

[RELEASE][DRAFT] TVM v0.6 Release candidate #4259

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Relay] Pass manager #2546

[Relay] Pass manager #2546

zhiics commented Feb 2, 2019 •

edited

Loading

tqchen commented Feb 2, 2019

zhiics commented Feb 2, 2019

wweic left a comment

tqchen commented Mar 1, 2019

zhiics commented Mar 2, 2019

zhiics commented Mar 2, 2019

tqchen commented Mar 2, 2019

tqchen Mar 3, 2019

tqchen left a comment

zhiics commented Mar 3, 2019 •

edited

Loading

wweic left a comment

tqchen left a comment

zhiics commented Mar 8, 2019

zhiics commented Mar 10, 2019

tqchen Mar 10, 2019

zhiics Mar 11, 2019 •

edited

Loading

tqchen Mar 10, 2019

zhiics Mar 11, 2019

tqchen Mar 10, 2019

jroesch Mar 11, 2019

zhiics Mar 11, 2019 •

edited

Loading

tqchen left a comment

tqchen Mar 11, 2019

tqchen Mar 11, 2019

zhiics Mar 11, 2019

tqchen Mar 11, 2019

tqchen Mar 11, 2019

		disabled)


		def create_module_pass(pass_name, opt_level, pass_func, required=None):

		disabled)


		def module_pass(opt_level, name=None, required=None):

		return create_function_pass


		def sequential_pass(opt_level, passes, name="sequential_pass", required=None,

[Relay] Pass manager #2546

[Relay] Pass manager #2546

Conversation

zhiics commented Feb 2, 2019 • edited Loading

tqchen commented Feb 2, 2019

zhiics commented Feb 2, 2019

wweic left a comment

Choose a reason for hiding this comment

tqchen commented Mar 1, 2019

zhiics commented Mar 2, 2019

zhiics commented Mar 2, 2019

tqchen commented Mar 2, 2019

Choose a reason for hiding this comment

tqchen left a comment

Choose a reason for hiding this comment

zhiics commented Mar 3, 2019 • edited Loading

wweic left a comment

Choose a reason for hiding this comment

tqchen left a comment

Choose a reason for hiding this comment

zhiics commented Mar 8, 2019

zhiics commented Mar 10, 2019

Choose a reason for hiding this comment

zhiics Mar 11, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zhiics Mar 11, 2019 • edited Loading

Choose a reason for hiding this comment

tqchen left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zhiics commented Feb 2, 2019 •

edited

Loading

zhiics commented Mar 3, 2019 •

edited

Loading

zhiics Mar 11, 2019 •

edited

Loading

zhiics Mar 11, 2019 •

edited

Loading