Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🐛 [Bug] Numerical error with dynamic shape support for aten.linear #3022

Closed
chohk88 opened this issue Jul 19, 2024 · 0 comments · Fixed by #3011
Closed

🐛 [Bug] Numerical error with dynamic shape support for aten.linear #3022

chohk88 opened this issue Jul 19, 2024 · 0 comments · Fixed by #3011
Assignees
Labels
bug Something isn't working

Comments

@chohk88
Copy link
Collaborator

chohk88 commented Jul 19, 2024

Bug Description

In the recent PR (#3011), we added dynamic shape support for the aten.linear operation. During testing, an assertion error was encountered when using torch.randn for weight initialization. This issue needs to be addressed and resolved.

Error Message:

AssertionError: Tensor-likes are not close! 
Mismatched elements: 323 / 2304 (14.0%)
Greatest absolute difference: 0.015128374099731445 at index (1, 1, 139) (up to 0.001 allowed) 
Greatest relative difference: 3.242828845977783 at index (1, 1, 154) (up to 0.001 allowed)

The error suggests a numerical discrepancy, with mismatched elements and significant differences in some tensor values.
No such error occurs when weights are initialized using torch.rand.

Reproduce

In the current dynamic shape test, weights are generated using self.weight = torch.rand(weight_shape). Changing this line to self.weight = torch.randn(weight_shape) causes the aforementioned error to occur.

@chohk88 chohk88 added the bug Something isn't working label Jul 19, 2024
@chohk88 chohk88 self-assigned this Jul 19, 2024
@chohk88 chohk88 linked a pull request Jul 19, 2024 that will close this issue
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant