Attention-augmented-Convolutional-Neural-Networks

As part of my research on creating attention based modules for improving accuracies on Computer Vision-tasks this is a initial experimental code that i have done I am utilizing the idea of self attention mechanism from the 2017 paper that states attention is all you need. More detailed Code with explanation will be made availale soon as our experiments conclude. Link to the 2017 paper attention is all you need-https://arxiv.org/pdf/1706.03762.pdf. Link to 2019 paper by Google brain on Attention augmented Convolutional neural Networks they are using two dimensional relative positional encodings to introduce translational inavariance along the height and width of the image, the idea is nice but there are certain downsides to it and right now my aim is to attempt to solve those downsides.-https://arxiv.org/abs/1904.09925