Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow uers to specify the name of moving mean and variance in batch_norm interface. #8069

Merged
merged 1 commit into from
Feb 2, 2018

Conversation

qingqing01
Copy link
Contributor

Fix #8068

@qingqing01 qingqing01 changed the title Allow uers to specify the name of moving mean and variance in batch_norm APU. Allow uers to specify the name of moving mean and variance in batch_norm interface. Feb 2, 2018
chengduoZH
chengduoZH previously approved these changes Feb 2, 2018
Copy link
Contributor

@chengduoZH chengduoZH left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!
V2 supports share parameters by name.

@@ -1508,13 +1510,15 @@ def batch_norm(input,
attr=helper.bias_attr, shape=param_shape, dtype=dtype, is_bias=True)

mean = helper.create_global_variable(
name=helper.moving_mean_name,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

helper.moving_mean_name ==> moving_mean_name

helper doesn't have this property.

class LayerHelper(object):
def __init__(self, layer_type, **kwargs):
self.kwargs = kwargs
self.layer_type = layer_type
name = self.kwargs.get('name', None)
if name is None:
self.kwargs['name'] = unique_name(self.layer_type)
@property
def name(self):
return self.kwargs['name']
@property
def main_program(self):
return default_main_program()
@property
def startup_program(self):
return default_startup_program()
def append_op(self, *args, **kwargs):
return self.main_program.current_block().append_op(*args, **kwargs)
def multiple_input(self, input_param_name='input'):
inputs = self.kwargs.get(input_param_name, [])
type_error = TypeError(
"Input of {0} layer should be Variable or sequence of Variable".
format(self.layer_type))
if isinstance(inputs, Variable):
inputs = [inputs]
elif not isinstance(inputs, list) and not isinstance(inputs, tuple):
raise type_error
else:
for each in inputs:
if not isinstance(each, Variable):
raise type_error
return inputs
def input(self, input_param_name='input'):
inputs = self.multiple_input(input_param_name)
if len(inputs) != 1:
raise "{0} layer only takes one input".format(self.layer_type)
return inputs[0]
@property
def param_attr(self):
return ParamAttr.to_attr(self.kwargs.get('param_attr', None))
@property
def bias_attr(self):
return ParamAttr.to_attr(self.kwargs.get('bias_attr', None))
def multiple_param_attr(self, length):
param_attr = self.param_attr
if isinstance(param_attr, ParamAttr):
param_attr = [param_attr]
if len(param_attr) != 1 and len(param_attr) != length:
raise ValueError("parameter number mismatch")
elif len(param_attr) == 1 and length != 1:
tmp = [None] * length
for i in xrange(length):
tmp[i] = copy.deepcopy(param_attr[0])
param_attr = tmp
return param_attr
def iter_inputs_and_params(self, input_param_name='input'):
inputs = self.multiple_input(input_param_name)
param_attrs = self.multiple_param_attr(len(inputs))
for ipt, param_attr in itertools.izip(inputs, param_attrs):
yield ipt, param_attr
def input_dtype(self, input_param_name='input'):
inputs = self.multiple_input(input_param_name)
dtype = None
for each in inputs:
if dtype is None:
dtype = each.dtype
elif dtype != each.dtype:
raise ValueError("Data Type mismatch: %d to %d" %
(dtype, each.dtype))
return dtype
def _create_weight_normalize(self, attr, shape, dtype):

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@chengduoZH

In line 1488:

helper = LayerHelper('batch_norm', **locals())
 class LayerHelper(object): 
     def __init__(self, layer_type, **kwargs): 
         self.kwargs = kwargs 

So helper.moving_mean_name is no problem.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@chengduoZH Done. change helper.moving_mean_name to moving_mean_name

@chengduoZH chengduoZH dismissed their stale review February 2, 2018 03:58

helper doesn't moving_mean_name .

@@ -1478,7 +1478,9 @@ def batch_norm(input,
param_attr=None,
bias_attr=None,
data_layout='NCHW',
name=None):
name=None,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if this name is still used?

Copy link
Member

@jacquesqiao jacquesqiao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@qingqing01 qingqing01 merged commit 4e7e39b into PaddlePaddle:develop Feb 2, 2018
@qingqing01 qingqing01 deleted the bn_name branch March 7, 2018 12:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants