-
Notifications
You must be signed in to change notification settings - Fork 5.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Evaluator accumulate #4828
Evaluator accumulate #4828
Conversation
|
||
class TestEvaluator(unittest.TestCase): | ||
def setup(self, scope, inputs, outputs): | ||
def __create_var__(var_name, arr): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
def __create_var__
-> def _create_var
.
class Evaluator(object): | ||
def __init__(self, | ||
scope, | ||
operator='accuracy', |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
看了下这个Evaluator
里只有对多个mini-batch间的结果累加求平均。但对于PrecisionRecall这种多个mini-batch间结果统计并不是简单的累加求平均。
所以感觉如果在Python里抽象出Evaluator
,用来统计多个mini-batch间的计算结果,需要抽象出一个Evaluator
基类接口,然后有很多子类,AccuracyEvalutor, PrecisionRecallEvalutor等等。
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
嗯,前面定义了avg_accumulate
,是解决这个问题的。但的确没有考虑编译时。需要把avg accumulator等作为op,才可以使python定义的计算是编译时的。
@qingqing01 @typhoonzero @wangkuiyi from my understanding the PaddlePaddle program is divided into compile time (currently Python) and runtime (C++): the Python code generates a protobuf which describes the computation, and the C++ code runs the computation description. This implementation of the executor is in Python (uses |
@helinwang you are right, I'll try to put this implementation to compile time. |
var_map[input] = [input] | ||
var_map[label] = [label] | ||
var_map[output] = [output] | ||
self.op = op.Operator(operator, **var_map) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Currently, we can not create operator immediately out of Python Block
and Program
.
Paddle/python/paddle/v2/framework/framework.py
Lines 430 to 446 in 1f11f77
class Program(object): | |
def __init__(self): | |
self.desc = core.ProgramDesc() | |
self.blocks = [Block(self, 0)] | |
self.current_block_idx = 0 | |
def __str__(self): | |
protostr = self.desc.serialize_to_string() | |
proto = framework_pb2.ProgramDesc.FromString(str(protostr)) | |
return proto.__str__() | |
def clone(self): | |
p = Program() | |
p.desc = core.ProgramDesc(self.desc) | |
p.blocks = [Block(p, i) for i in xrange(self.desc.num_blocks())] | |
p.sync_with_cpp() | |
return p |
May I just merge this work into codebase and let me polish it to fit in the refactorization code? : )
Our book porting work need this feature eagerly. @typhoonzero @qingqing01
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure, Thanks. Please also refer to some of my thoughts: #5157
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
I just merge this work into codebase and let me polish it.
Our book chapters need this feature eagerly.
@typhoonzero @helinwang
Fix #3808