-
Notifications
You must be signed in to change notification settings - Fork 356
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix: Layer norm Torchscript converter #3062
Conversation
2623220
to
1eebc04
Compare
e3bb75d
to
3e9b73f
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, minor comments
gamma = tensor_to_const(ctx, gamma_torch_tensor); | ||
} else { | ||
gamma = args[2].ITensorOrFreeze(ctx); | ||
gamma = broadcast(ctx, n, gamma, input_shape_vec.size(), "gamma"); | ||
// gamma = broadcast(ctx, n, gamma, input_shape_vec.size(), "gamma"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
comment can be removed
beta = tensor_to_const(ctx, beta_torch_tensor); | ||
} else { | ||
beta = args[3].ITensorOrFreeze(ctx); | ||
beta = broadcast(ctx, n, beta, input_shape_vec.size(), "beta"); | ||
// beta = broadcast(ctx, n, beta, input_shape_vec.size(), "beta"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
comment can be removed
3e9b73f
to
2878e39
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are some changes that do not conform to C++ style guidelines:
diff --git a/home/runner/work/TensorRT/TensorRT/core/runtime/execute_engine.cpp b/tmp/changes.txt
index d21aa5e..6b75397 100644
--- a/home/runner/work/TensorRT/TensorRT/core/runtime/execute_engine.cpp
+++ b/tmp/changes.txt
@@ -351,16 +351,16 @@ std::vector<at::Tensor> execute_engine(std::vector<at::Tensor> inputs, c10::intr
// If in CUDAGraph mode, results need to be copied to the result buffers (on caller stream)
for (size_t o = 0; o < compiled_engine->output_buffers.size(); o++) {
outputs[o].copy_(compiled_engine->output_buffers[o], false);
- }
+ }
- if (compiled_engine->profile_execution) {
- LOG_INFO(std::endl << *compiled_engine->trt_engine_profiler);
- dump_trace(compiled_engine->trt_engine_profile_path, *compiled_engine->trt_engine_profiler);
- compiled_engine->dump_engine_layer_info();
- }
+ if (compiled_engine->profile_execution) {
+ LOG_INFO(std::endl << *compiled_engine->trt_engine_profiler);
+ dump_trace(compiled_engine->trt_engine_profile_path, *compiled_engine->trt_engine_profiler);
+ compiled_engine->dump_engine_layer_info();
+ }
- return outputs;
-}
+ return outputs;
+ }
} // namespace runtime
} // namespace core
ERROR: Some files do not conform to style guidelines
2878e39
to
ee21db0
Compare
Signed-off-by: Naren Dasan <naren@narendasan.com> Signed-off-by: Naren Dasan <narens@nvidia.com>
Signed-off-by: Naren Dasan <naren@narendasan.com> Signed-off-by: Naren Dasan <narens@nvidia.com>
ee21db0
to
8dce1e6
Compare
BERT test is failing because of data dependent intermediate values which aren't supported in the TS frontend Signed-off-by: Naren Dasan <naren@nvidia.com> Signed-off-by: Naren Dasan <naren@narendasan.com>
8dce1e6
to
0980096
Compare
Description
Fixes the broadcast implementation from the layer norm converter in TorchScript
Fixes # (issue)
Type of change
Please delete options that are not relevant and/or add your own.
Checklist: