Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Polishing the embedding layer and the fc layer documentation #6806

Merged
merged 2 commits into from
Dec 21, 2017
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
99 changes: 60 additions & 39 deletions python/paddle/v2/fluid/layers/nn.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,32 +25,48 @@ def fc(input,
act=None,
name=None):
"""
Fully Connected Layer.
**Fully Connected Layer**

This layer accepts multiple inputs and applies a linear transformation to each input.
If activation type is provided, the corresponding non-linear function is applied to the
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor: Did you mean "activation" instead of "non-linear" in 'the corresponding non-linear function'?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 5d3131c

output of the linear transformation. For each input :math:`X`, the equation is:

.. math::

Out = Act(WX + b)

In the above equation:

* :math:`X`: Input value, a tensor with rank at least 2.
* :math:`W`: Weight, a 2-D tensor with shape [M, N].
* :math:`b`: Bias, a 2-D tensor with shape [M, 1].
* :math:`Act`: Activation function.
* :math:`Out`: Output value, same shape with :math:`X`.

All the input variables are passed in as local variables to the LayerHelper
constructor.

Args:
input: The input tensor to the function
size: The size of the layer
num_flatten_dims: Number of columns in input
param_attr: The parameters/weights to the FC Layer
param_initializer: Initializer used for the weight/parameter. If None, XavierInitializer() is used
bias_attr: The bias parameter for the FC layer
bias_initializer: Initializer used for the bias. If None, then ConstantInitializer() is used
act: Activation to be applied to the output of FC layer
name: Name/alias of the function
main_program: Name of the main program that calls this
startup_program: Name of the startup program

This function can take in multiple inputs and performs the Fully Connected
function (linear transformation) on top of each of them.
So for input x, the output will be : Wx + b. Where W is the parameter,
b the bias and x is the input.

The function also applies an activation (non-linearity) on top of the
output, if activation is passed in the input.

All the input variables of this function are passed in as local variables
to the LayerHelper constructor.
input(Variable|list): Input tensors. Each tensor has a rank of atleast 2
size(int): Output size
num_flatten_dims(int): Number of columns in input
param_attr(ParamAttr|list): The parameters/weights to the FC Layer
bias_attr(ParamAttr|list): Bias parameter for the FC layer
act(str): Activation type
name(str): Name/alias of the function

Returns:
Variable: The tensor variable storing the transformation and \
non-linearity activation result.

Raises:
ValueError: If rank of input tensor is less than 2.

Examples:
.. code-block:: python

data = fluid.layers.data(name='data', shape=[32, 32], dtype='float32')
fc = fluid.layers.fc(input=data, size=1000, act="tanh")
"""
helper = LayerHelper('fc', **locals())

Expand Down Expand Up @@ -91,25 +107,30 @@ def fc(input,

def embedding(input, size, is_sparse=False, param_attr=None, dtype='float32'):
"""
Embedding Layer.
**Embedding Layer**

This layer is used to lookup a vector of IDs, provided by *input*, in a lookup table.
The result of this lookup is the embedding of each ID in the *input*.

All the input variables are passed in as local variables to the LayerHelper
constructor.

Args:
param_initializer:
input: The input to the function
size: The size of the layer
is_sparse: A flag that decleares whether the input is sparse
param_attr: Parameters for this layer
dtype: The type of data : float32, float_16, int etc
main_program: Name of the main program that calls this
startup_program: Name of the startup program

This function can take in the input (which is a vector of IDs) and
performs a lookup in the lookup_table using these IDs, to result into
the embedding of each ID in the input.

All the input variables of this function are passed in as local variables
to the LayerHelper constructor.
input(Variable): Input to the function
size(int): Output size
is_sparse(bool): Boolean flag that specifying whether the input is sparse
param_attr(ParamAttr): Parameters for this layer
dtype(np.dtype|core.DataType|str): The type of data : float32, float_16, int etc

Returns:
Variable: The tensor variable storing the embeddings of the \
supplied inputs.

Examples:
.. code-block:: python

data = fluid.layers.data(name='ids', shape=[32, 32], dtype='float32')
fc = fluid.layers.embedding(input=data, size=16)
"""

helper = LayerHelper('embedding', **locals())
Expand Down