Add uniform random operator #3293
Conversation
It can be run both CPU/GPU. configure attributes are:
* min: the min value of uniform random
* max: the max value of uniform random
* dims: the dimension of output tensor
* seed: the random seed of uniform random. 0 means generate a seed each
time.
reyoung
force-pushed
the
feature/uniform_random_op
branch
from
d946e18 to
e376bda
Compare
| @@ -38,9 +38,10 @@ cc_test(backward_test SRCS backward_test.cc DEPS backward) | |||
| cc_library(paddle_pybind SHARED | |||
| SRCS pybind.cc | |||
| DEPS pybind python backward | |||
| fc_op | |||
There was a problem hiding this comment.
Use space for indent.
reyoung
force-pushed
the
feature/uniform_random_op
branch
2 times, most recently
from
cfcb03b to
8a8f956
Compare
| #include <thrust/iterator/counting_iterator.h> | ||
| #include <thrust/random.h> | ||
| #include <thrust/transform.h> | ||
| #include "paddle/operators/type_alias.h" |
There was a problem hiding this comment.
may be we need to remove type_alias header declaration from now on.
|
|
||
| template <typename T> | ||
| struct UniformGenerator { | ||
| T min_, max_; |
There was a problem hiding this comment.
@reyoung found eigen built-in random generator easy to crash. we cannot point out the reason. should we put a TODO here?
Not a good news, we need to implement more generator in thrust by ourselves such as random_gama, ramdom_guassian
tf.random_normal
tf.truncated_normal
tf.random_uniform
tf.random_shuffle
tf.random_crop
tf.multinomial
tf.random_gamma
There was a problem hiding this comment.
Done. TODO is added.
In Eigen, only normal and uniform two generators are supported. i.e., Eigen is not good for randomizing a tensor.
| AddAttr<std::vector<int>>("dims", "the dimension of random tensor"); | ||
| AddAttr<float>("min", "Minimum value of uniform random").SetDefault(-1.0f); | ||
| AddAttr<float>("max", "Maximun value of uniform random").SetDefault(1.0f); | ||
| AddAttr<int>("seed", |
There was a problem hiding this comment.
can we bind the seed to the device context or environment?
In my view, the user may set a random seed before doing any experiment. He will not set a random seed in every random operator function calls.
There was a problem hiding this comment.
I think that logic should be handled in a higher level of Paddle.
In normal situation, user does not need set seed, just set them to zero, Paddle will generate a seed from std::random_device
| AddAttr<float>("min", "Minimum value of uniform random").SetDefault(-1.0f); | ||
| AddAttr<float>("max", "Maximun value of uniform random").SetDefault(1.0f); | ||
| AddAttr<int>("seed", | ||
| "Random seed of uniform random. " |
There was a problem hiding this comment.
further more, each stream in GPU needs a different seed.
There was a problem hiding this comment.
Each stream in GPU's seed is decided outside operator.
reyoung
force-pushed
the
feature/uniform_random_op
branch
from
56fb666 to
d7f0eb6
Compare
|
|
||
| namespace paddle { | ||
| namespace operators { | ||
| class RandomOp : public OperatorWithKernel { |
There was a problem hiding this comment.
RamdomOp is too general, maybe rename with UniformRandomOp is better? I have implemented a Gaussian random op.
| void InferShape(const InferShapeContext &ctx) const override { | ||
| PADDLE_ENFORCE(GetAttr<float>("min") < GetAttr<float>("max"), | ||
| "uniform_random's min must less then max"); | ||
| auto tensor = ctx.Output<Tensor>(0); |
There was a problem hiding this comment.
rename auto withauto* for clear.
| void InferShape(const framework::InferShapeContext& ctx) const override { | ||
| PADDLE_ENFORCE(GetAttr<float>("min") < GetAttr<float>("max"), | ||
| "uniform_random's min must less then max"); | ||
| auto tensor = ctx.Output<framework::Tensor>(0); |
There was a problem hiding this comment.
use auto * instead of auto for clear.
It can be run both CPU/GPU. configure attributes are:
time.