🐛 [Bug] Returning list of tensors fails when operations are applied to tensors: RuntimeError: [Error thrown at core/conversion/conversion.cpp:220] List type. Only a single tensor or a TensorList type is supported.
#899
Comments
chaoz-dev
changed the title
|
May be due to #428 |
|
Ah, I think I understand what is happening. In the cases of append and similar cases where PyTorch operates on groupings of Tensors which are already converted, we use a container class so that PyTorch and TensorRT can track data. Looks like we end up with a list of these container classes instead of just ITensors or Tensors and so the PyTorch type system treats it as a Generic List and not a TensorList. |
|
@peri044 I think we decided for this case to wait for collections right? |
Ah gotcha, yeah this makes sense, especially given the printed output: |
chaoz-dev
changed the title
RuntimeError: [Error thrown at core/conversion/conversion.cpp:220] List type. Only a single tensor or a TensorList type is supported.
|
I have a slightly different looking setup that triggers a similar problem. Is there a workaround? # tuple.py
import torch
import torch.nn as nn
import torch_tensorrt as torchtrt
import torch_tensorrt.logging as torchtrt_logging
torchtrt_logging.set_reportable_log_level(torchtrt_logging.Level.Info)
torch.manual_seed(0)
DEVICE = torch.device("cuda:0")
SHAPE = (1, 3, 100, 100)
class Model(torch.nn.Module):
def __init__(self):
super().__init__()
self.seq1 = nn.Sequential(nn.Conv2d(3, 6, 3, 1, 1), nn.ReLU(), nn.MaxPool2d(2, 2))
self.seq2 = nn.Sequential(nn.Conv2d(6, 6, 3, 1, 1), nn.ReLU(), nn.MaxPool2d(2, 2))
self.seq3 = nn.Sequential(nn.Conv2d(6, 6, 3, 1, 1), nn.ReLU(), nn.MaxPool2d(2, 2))
def forward(self, x):
a = self.seq1(x)
b = self.seq2(a)
c = self.seq3(b)
## It works fine without this
a1, a2 = b.split([5, 1], dim=1)
a1 = a1.max(dim=1, keepdim=True)[0]
b = torch.cat([a1, a2], dim=1)
## ^
return (b, c)
if __name__ == "__main__":
tensor = torch.randn(SHAPE, dtype=torch.float32, device=DEVICE)
model = Model().eval().to(DEVICE)
out = model(tensor)
print(f"Model: {out}")
model_trt = torchtrt.compile(
model,
inputs=[
torchtrt.Input(shape=SHAPE),
],
enabled_precisions={torch.float},
)
out_trt = model_trt(tensor)
print(f"Model TRT: {out_trt}")
assert torch.max(torch.abs(out[0] - out_trt[0])) < 1e-6
assert torch.max(torch.abs(out[1] - out_trt[1])) < 1e-6when I remove the highlighted part of the code, the constructed graph correctly builds a TupleConstruct like this But when that part is active, I get this error and the constructed graph is wrong Is there a way to manually remove the |
|
So I was looking around in the built If you call it just after the compilation, your graph becomes like this and it'll work. It's better to not have the |
|
meet same problem, any way to avoid this bug? |
|
Just call after compilation and it'll work. |
|
@ashafaei
this is the error: how could i call the function |
|
@pupumao Are you getting this error during compilation? The workaround I shared is for a different problem. The model gets compiled, but then during inference throws this error. You may have better luck if you use the |
narendasan
added
feature request
@narendasan is this a tested case by collections for v1.2? |
|
Yes I tested this and it works |
|
Closing. @chaoz-dev please reopen / file a new issue if there is still a problem. |
Bug Description
Returning a list of tensors fails when ops are applied to the tensors prior to appending them to the list that is returned.
This is not the case if tensors are directly appended to the list without applying any operations.
To Reproduce
Run the following:
This throws the following error:
Expected behavior
Graph should return a list of tensors without errors.
Environment
conda,pip,libtorch, source): CondaAdditional context
Note that changing the forward function to the following definition:
will succeed with the following output:
The text was updated successfully, but these errors were encountered: