Deep Learning

This is the online course on Udacity.

Instructor: Dr. Vincent Vanhoucke from Google Brain.

Notice: The original assignments can be found at this link.

* Logistic Regression, Stochastic Optimization, and Data & parameters tuning.

notMNIST - Deep Learning with Tensorflow.

* Deep Neural Networks

Implemented a fully connected network using SGD and ReLUs with Tensorflow

* Regularization

Implemented a multi-layer neural network using ReLUs, L2-regularization, and dropout, to prevent overfitting.

* Convolutional Networks

Reading:
- C. Olah, Conv Nets: A Modular Perspective.
- C. Olah, Understanding Convolutions.
- C. Olah, Calculus on Computational Graphs: Backpropagation.
- V. Dumoulin and F. Visin, A guide to convolution arithmetic for deep learning. (arXiv, March 2016).
Implemented a convolutional neural network.

* Deep Models for Text and Sequences

[Embeddings]

Reading:
- T. Mikolov et al., Efficient Estimation of Word Representations in Vector Space. (arXiv, Sep 2013).
- T. Mikolov et al., Distributed Representations of Words and Phrases and their Compositionality. (arXiv, Oct 2013). Subsampling and Negative sampling in Word2Vec.
Implemented a Word2Vec with the skip-gram model and CBOW model in different datasets of Wikipedia text as follows:
- With text8.zip dataset.
- With enwik8.zip dataset.

[Recurrent Neural Networks]

Reading:
- Understanding LSTM networks
- Attention and Augmented Recurrent Neural Networks
- H. Sak et al., Long Short-Term Memory Based Recurrent Neural Network Architectures for Large Vocabulary Speech Recognition. (arXiv, Feb 2014).
LSTM: Long Short-Term Memory
- Note: Backprop for RNNs example can be found at these links [1] and [2].
- char-LSTM implementation with numpy

[Resources]

Deep Learning with Python, Nov 2017 by François Chollet