Code for our paper: "A high-performance neuroprosthesis for speech decoding and avatar control" (https://www.nature.com/articles/s41586-023-06443-4)
To explore the code for each modality, check out the corresponding folder!
text folder - train a model on the 1024-word-General sentence set and evaluate it on completely unseen sentences!
synthesis - train a model using HuBERT units for speech synthesis from brain activity
avatar - train a model on orofacial movements
Thank you for your interest in our work!