sh pretrain.sh

[YePG]

RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [90, 128, 1536]], which is output 0 of AddBackward0, is at version 2; expected version 1 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).

How to solve this problem?

[YePG]

q *= self.scaling
Traceback (most recent call last):
File "pretrain.py", line 277, in
main(args, 3)
File "pretrain.py", line 234, in main
loss.backward()
File "/home/pgye/anaconda3/envs/ypg/lib/python3.6/site-packages/torch/tensor.py", line 221, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph)
File "/home/pgye/anaconda3/envs/ypg/lib/python3.6/site-packages/torch/autograd/init.py", line 132, in backward
allow_unreachable=True) # allow_unreachable flag
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [90, 128, 1536]], which is output 0 of AddBackward0, is at version 2; expected version 1 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).

[YePG]

ok ! I solve it by upgrade torch=1.8