Implements Gumbel Softmax VAE for Semi Supervised Learning.
From: http://arxiv.org/abs/1611.01144
Repo Structure from: https://github.com/jxmorris12/vec2text/
- NLP repos are smartly organized. Could be useful to adapt them for statistical ML purposes.
Additional experiments.
- What if we replace VAE loss with Contrastive Divergence Loss from: https://arxiv.org/pdf/1905.04062
- Clever straight-through estimator (Reinmax) due to Heun's methods; from: https://arxiv.org/pdf/2304.08612
References:
(1) Jang, Eric, Shixiang Gu, and Ben Poole. "Categorical reparameterization with gumbel-softmax." arXiv preprint arXiv:1611.01144 (2016).
(2) Ruiz, Francisco, and Michalis Titsias. "A contrastive divergence for combining variational inference and mcmc." International Conference on Machine Learning. PMLR, 2019.
(3) Morris, John X., et al. "Text embeddings reveal (almost) as much as text." arXiv preprint arXiv:2310.06816 (2023).
(4) Liu, Liyuan, et al. "Bridging discrete and backpropagation: Straight-through and beyond." Advances in Neural Information Processing Systems 36 (2024).