Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fp16] support fp16 on AlphaDropout #50917

Merged
merged 1 commit into from
Feb 27, 2023
Merged

[fp16] support fp16 on AlphaDropout #50917

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions python/paddle/fluid/tests/unittests/test_dropout_op.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -980,6 +980,30 @@ def test_dygraph(self):
result.numpy(), result_np, rtol=1e-05
)

def test_static_fp16_gpu(self):
if paddle.fluid.core.is_compiled_with_cuda():
place = paddle.CUDAPlace(0)
with paddle.static.program_guard(
paddle.static.Program(), paddle.static.Program()
):
input = np.random.random([2, 3]).astype("float16")

x = paddle.static.data(name="x", shape=[2, 3], dtype="float16")

m = paddle.nn.AlphaDropout(p=0.0)
y = m(x)

exe = paddle.static.Executor(place)
res = exe.run(
paddle.static.default_main_program(),
feed={
"x": input,
},
fetch_list=[y],
)

np.testing.assert_allclose(res[0], input, rtol=1e-05)


class TestDropoutWithDeterminateSeedGenerator(unittest.TestCase):
def setUp(self):
Expand Down
4 changes: 2 additions & 2 deletions python/paddle/nn/functional/common.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1384,7 +1384,7 @@ def alpha_dropout(x, p=0.5, training=True, name=None):
Alpha Dropout fits well to SELU activate function by randomly setting activations to the negative saturation value.

Args:
x (Tensor): The input tensor. The data type is float32 or float64.
x (Tensor): The input tensor. The data type is float16, float32 or float64.
p (float | int): Probability of setting units to zero. Default 0.5.
training (bool): A flag indicating whether it is in train phrase or not. Default True.
name (str, optional): Name for the operation (optional, default is None). For more information, please refer to :ref:`api_guide_Name`.
Expand Down Expand Up @@ -1416,7 +1416,7 @@ def alpha_dropout(x, p=0.5, training=True, name=None):

if not in_dynamic_mode():
check_variable_and_dtype(
x, 'x', ['float32', 'float64'], 'alpha_dropout'
x, 'x', ['float16', 'float32', 'float64'], 'alpha_dropout'
)

if training:
Expand Down