Add Embedding Quantization to QAT module_swap flow #886
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/886
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit 1430e0e with merge base a4221df ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
facebook-github-bot
added
the
CLA Signed
|
This pull request was exported from Phabricator. Differential Revision: D62664322 |
|
This pull request was exported from Phabricator. Differential Revision: D62664322 |
Summary: Pull Request resolved: pytorch#886 Adding the embedding quantizer in the same fashion as the other module swap setup. Differential Revision: D62664322
TiRune
force-pushed
the
export-D62664322
branch
from
0ac8d53
to
5259084
Compare
|
This pull request was exported from Phabricator. Differential Revision: D62664322 |
Summary: Pull Request resolved: pytorch#886 Adding the embedding quantizer in the same fashion as the other module swap setup. Differential Revision: D62664322
TiRune
force-pushed
the
export-D62664322
branch
from
5259084
to
a087e50
Compare
| @@ -965,6 +965,41 @@ def forward(self, input: torch.Tensor) -> torch.Tensor: | |||
| self.precision, | |||
| ) | |||
|
|
|||
|
|
|||
| def _replace_embedding_4w( | |||
There was a problem hiding this comment.
I'm wondering if this can be added at the user code side, since we are planning to deprecate the module swap API
There was a problem hiding this comment.
Please don't deprecate the module swap API - it's the easiest to work with and extend.
I'll likely have a headache if I wanted to make things work quickly and effectively with the tensor subclass stuff.
If you guys have a few minutes, we can discuss together how to add this to the tensor subclass stuff as well... but...
There was a problem hiding this comment.
OK, I think keeping multiple implementations of the same thing might be confusing, we can gather all the requirements and decide on the long term plan I think, I'm asking Andrew to take a stab first
Summary: Pull Request resolved: pytorch#886 Adding the embedding quantizer in the same fashion as the other module swap setup. Differential Revision: D62664322
|
This pull request was exported from Phabricator. Differential Revision: D62664322 |
TiRune
force-pushed
the
export-D62664322
branch
from
a087e50
to
1430e0e
Compare
Summary: Adding the embedding quantizer in the same fashion as the other module swap setup.
Differential Revision: D62664322