In this project, there is a comparison of classic RNN model with the task of sentiment analysis,
and RNN supplemented with Multihead attention.
In addition, there is an examination of different RNN based architectures and different hyper-parameters
for different sentiment analysis based tasks, and performance comparison.
The two datasets used for the task are Stanford Sentiment Treebank for 3 and 5 sentiment classes (SST-5 is 'fine grained' when regarding sentiment labeling).
The two datasets are considered difficult where SST-5 has State of the art performance of ~50% accuracy, and SST-3 with ~70%.
This allows a good comparison over perfomance with noticable improvements for the attention based model.
Another performance comparison was about the confusion of the different models, trying to visualize differences
between the performances of different models for different classes and with different statistical properties.
The models results can be viewed in Project.ipynb, and the hyperparameters search results can be viewed in Hyper_Parameters.ipynb.
The models code can be found under the project/ folder