Latent diff turned larger after implicit regularization fine-tunning

[txytju]

Thanks for your great work! I have tested your ckpt and on my model tuned using implicit regularization, and find that latent diff turned larger after implicit regularization fine-tunning. Can you help me with that?

The way I test is like this: (E(aug(x)) - aug(E(x))).abs().mean()