Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

Use single-bit for mask in dropout operator #16735

Open
wants to merge 75 commits into
base: master
Choose a base branch
from
Open

Use single-bit for mask in dropout operator #16735

Show file tree
Hide file tree
Changes from 66 commits
Commits
Show all changes
75 commits
Select commit Hold shift + click to select a range
d9b7ee1
basic version that is verfied on CPU
apeforest Nov 6, 2019
5ae7bbe
add log message and TODO
apeforest Nov 6, 2019
0b22e01
add backward support for 1-bit mask
apeforest Dec 22, 2019
10ab0c4
fix the race condition in LaunchRNG
apeforest Dec 23, 2019
d56451f
refactoring to improve readability
apeforest Dec 23, 2019
a05ca5c
address reviewer comment and test w/o cudnn
apeforest Dec 26, 2019
ba9eada
remove check from kernel
apeforest Dec 26, 2019
75014ca
fix compile error when index_t is int64_t
apeforest Dec 26, 2019
512569d
fix unit test
apeforest Dec 28, 2019
3ba54f1
use faster operation to replace modulo
apeforest Dec 28, 2019
d4369ec
replace modulo
apeforest Dec 28, 2019
36156d9
replace div with bitwise shift in kernel
apeforest Jan 2, 2020
0c1c83c
fix cpp unit test
apeforest Jan 3, 2020
02be788
fix dropout perf cpp test
apeforest Jan 9, 2020
1766a50
fix cpp test
apeforest Jan 10, 2020
6319779
fix a unit test
apeforest Jan 10, 2020
75f6c01
fix unit test
apeforest Jan 15, 2020
a4e6db0
basic version that is verfied on CPU
apeforest Nov 6, 2019
ce26bea
add log message and TODO
apeforest Nov 6, 2019
7716ca5
add backward support for 1-bit mask
apeforest Dec 22, 2019
9d9042f
fix the race condition in LaunchRNG
apeforest Dec 23, 2019
616710b
refactoring to improve readability
apeforest Dec 23, 2019
ffb213f
address reviewer comment and test w/o cudnn
apeforest Dec 26, 2019
820643b
remove check from kernel
apeforest Dec 26, 2019
4fe6978
fix compile error when index_t is int64_t
apeforest Dec 26, 2019
3c9db6c
fix unit test
apeforest Dec 28, 2019
2cc7db1
use faster operation to replace modulo
apeforest Dec 28, 2019
524dc67
replace modulo
apeforest Dec 28, 2019
dd2144e
replace div with bitwise shift in kernel
apeforest Jan 2, 2020
8b1a6cf
fix cpp unit test
apeforest Jan 3, 2020
bc45f43
fix dropout perf cpp test
apeforest Jan 9, 2020
c912dce
fix cpp test
apeforest Jan 10, 2020
8bf911e
fix a unit test
apeforest Jan 10, 2020
ec4e60a
fix unit test
apeforest Jan 15, 2020
0ea3507
fix lint
apeforest Jan 16, 2020
2d73ae5
basic version that is verfied on CPU
apeforest Nov 6, 2019
f2a9232
add log message and TODO
apeforest Nov 6, 2019
7019e35
add backward support for 1-bit mask
apeforest Dec 22, 2019
8eb5fae
fix the race condition in LaunchRNG
apeforest Dec 23, 2019
3dfd060
refactoring to improve readability
apeforest Dec 23, 2019
971b0bb
address reviewer comment and test w/o cudnn
apeforest Dec 26, 2019
6c162bc
remove check from kernel
apeforest Dec 26, 2019
965d655
fix compile error when index_t is int64_t
apeforest Dec 26, 2019
d7572ec
fix unit test
apeforest Dec 28, 2019
c39092f
use faster operation to replace modulo
apeforest Dec 28, 2019
1ae64f0
replace modulo
apeforest Dec 28, 2019
b443fc4
replace div with bitwise shift in kernel
apeforest Jan 2, 2020
b257ce5
fix cpp unit test
apeforest Jan 3, 2020
b4681f2
fix dropout perf cpp test
apeforest Jan 9, 2020
5c6a4a9
fix cpp test
apeforest Jan 10, 2020
8c6c8a6
fix a unit test
apeforest Jan 10, 2020
6e0a6b5
fix unit test
apeforest Jan 15, 2020
5bc583d
fix lint
apeforest Jan 16, 2020
0302527
Merge branch 'perf/dropout-mask' of https://github.com/apeforest/incu…
apeforest Jan 30, 2020
7b5262e
Merge remote-tracking branch 'upstream/master' into perf/dropout-mask
apeforest Jan 30, 2020
a5e0b1d
Merge branch 'perf/dropout-mask' of https://github.com/apeforest/incu…
apeforest Jan 30, 2020
f2b2d2a
Merge remote-tracking branch 'upstream/master' into perf/dropout-mask
apeforest Feb 3, 2020
f15a7a5
Fix bug in MKL Dropout
apeforest Feb 7, 2020
b59cc77
Remove unnecessary check of MKL availability
apeforest Feb 7, 2020
90b16d2
Merge remote-tracking branch 'upstream/master' into perf/dropout-mask
apeforest Feb 7, 2020
16f3e2a
Fix lint
apeforest Feb 7, 2020
183c10d
Remove unused header
apeforest Feb 7, 2020
99ad39b
Fix CI build error
apeforest Feb 10, 2020
4aafc95
Speedup forward and backward kernel in MKL Dropout
apeforest Feb 10, 2020
cf0d95e
Improve omp for loop performance
apeforest Feb 11, 2020
c5188ab
Merge remote-tracking branch 'upstream/master' into perf/dropout-mask
apeforest Feb 12, 2020
354d83c
Speed up backward compute in CPU
apeforest Feb 14, 2020
9a231b1
Merge remote-tracking branch 'upstream/master' into perf/dropout-mask
apeforest Feb 14, 2020
013bb48
Improve speed of backward
apeforest Feb 14, 2020
2183a23
Remove unncessary block in backward
apeforest Feb 14, 2020
746a8f0
Do not use bit-mask when MKL dropout is used.
apeforest Feb 19, 2020
93a381a
Merge remote-tracking branch 'upstream/master' into perf/dropout-mask
apeforest Feb 19, 2020
2b1bf66
Merge remote-tracking branch 'upstream/master' into perf/dropout-mask
apeforest Feb 21, 2020
469e3b9
Revert "Do not use bit-mask when MKL dropout is used."
apeforest Feb 22, 2020
575cd2f
Merge remote-tracking branch 'upstream/master' into perf/dropout-mask
apeforest Feb 26, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions src/operator/mshadow_op.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -538,8 +538,8 @@ MXNET_UNARY_MATH_OP(square, math::sqr(a));
MXNET_UNARY_MATH_OP(square_grad, 2.0f * math::id(a));

/*! \brief used for generate Bernoulli mask */
MXNET_BINARY_MATH_OP_NC(threshold, a < b ? DType(1) : DType(0));
MXNET_BINARY_MATH_OP_NC(threshold_eq, a <= b ? DType(1) : DType(0));
MXNET_BINARY_LOGIC_OP_NC(threshold, a < b ? DType(1) : DType(0));
MXNET_BINARY_LOGIC_OP_NC(threshold_eq, a <= b ? DType(1) : DType(0));

/*! \brief used for generate element of abs */
MXNET_UNARY_MATH_OP(abs, math::fabs(a)); // NOLINT(*)
Expand Down
Loading