Quantisation for TTS models

[ApoorveK]

So, the main idea behind this discussion is finding efficient way to Quantize TTS models, for faster inferences. Since the layers being used actually are custom layers (for example in VITS model) so how can we Quantize these layers and which framework would be ideal to do that. Have been through Pytorch documentations, and I am not able to find about "Fusing Custom Layers" which is important step in quantisation of TTS model/s (whether it is Quantized Aware Training or Post Training Quantisation).

[Nanayeb34]

I think one of the ways to apply pytorch's dynamic quantize will be to instantiate the model class. At present, I can't seem to find the model class of the architectures. It looks like we would have to scrap it from the respective .py files of the models

[DrewThomasson]

I've been trying to quantize the xtts model

Ive gotten to quantize them but inference seems to be broken for all fo them :/

Where any work related to this will be located:

https://github.com/DrewThomasson/Quantize_xtts/tree/main