Skip to content

Latest commit

 

History

History
197 lines (180 loc) · 5.57 KB

deepLearning.md

File metadata and controls

197 lines (180 loc) · 5.57 KB

AI-Deep-Learning

Starred Links

Introduction to Deep Learning

  • Definition
    • Neural Network Overview
    • Neural Network Representation
    • Why Deep Representations?
    • What does this have to do with the brain?
  • History of Deep Learning

Basics of Neural Network

  • Binary Classification
  • Logistic Regression
  • Gradient Descent
    • Derivatives
    • More Derivative Examples
    • Logistic Regression Gradient Descent
    • Gradient Descent on m Examples
    • Gradient descent for neural networks
  • Computation Graph
    • Dervatives with a Computation Graph
  • Vectorization
    • More Examples of Vectorization
    • Vectorizing Logistic Regresstion
    • Vectorizing Logistic Regression's Gradient
    • Vectorizing across multiple examples
    • Justification for vectorized implementation
  • Broadcasting in Python
  • Activation Functions
    • Why need a nonlinear activation function
    • Derivatives of activation functions
  • Cost Function
    • Explanation of logistic regression cost function
  • Forward Propagation and Backward Propagation
    • Computing a Neural Network's output
    • Backpropagation intuition
    • Forward propagation in a Deep Network
  • Building Deep Neural Networks
    • Deep L-layer neural network
    • Building Blocks of Deep Neural Networks

Regularization for Deep Learning

  • Tain/ Dev /Test Sets
  • Bias / Variance
  • Basic Recipe for Machine Learning
  • Regularization
  • Why Regularization Reduces Overfitting
  • Dropout Regularization
  • Understanding Dropout
  • Other Regularization Methods
  • Normalizing Inputs
  • Vanishing / Exploding Gradients
  • Weight Initialization for Deep Networks
  • Numerical Approximation of Gradients
  • Gradient Checking

Optimization for Training Deep Models

  • Getting your Matrix Dimensions Right
  • Random Initialization
  • Mini-batch Gradient Descent
  • Understanding Mini-batch Gradient Descent
  • Exponentially Weighted Averages
  • Understanding Exponentially Weighted Averages
  • Bias Correction in Exponentially Weighted Averages
  • Gradient Descent with Momentum
  • RMSprop
  • Adam Optimization Algorithm
  • Learning Rate Decay
  • The Problem of Local Optima

Hyperparameter Tunning

  • Parameters vs Hyperparameters
  • Tuning Process
  • Using an Appropriate Scale to pick Hyperparameters
  • Hyperparameters Tuning in Practice: Pandas vs. Caviar
  • Normalizing Activations in a Network
  • Fitting Batch Norm into a Neural Network
  • Why does Batch Norm work?
  • Batch Norm at Test Time
  • Softmax Regression
  • Training a Softmax Classifier

Deep Learning Frameworks

  • Deep Learning Frameworks
  • TensorFlow

Convolutional Neural Networks

  • Foundations of Convolutional Neural Networks
    • Computer Vision
    • Edge Detection Example
    • More Edge Detection
    • Padding
    • Strided Convolutions
    • Convolutions Over Volume
    • One Layer of a Convolutional Network
    • Simple Convolutional Network Example
    • Pooling Layers
    • CNN Example
    • Why Convolutions?
  • Deep Convolutional Models: Case Studies
    • Why look at case studies?
    • Classic Networks
    • ResNets
    • Why ResNets Work?
    • Networks in Networks and 1x1 Convolutions
    • Inception Network Motivation
    • Inception Network
    • MobileNet
    • MobileNet Architecture
    • EfficientNet
    • Using Open-Source Implementation
    • Transfer Learning
    • Data Augmentation
    • State of Computer Vision
  • Object Detection
    • Object Localization
    • Landmark Detection
    • Object Detection
    • Convolutional Implementation of Sliding Windows
    • Bounding Box Predictions
    • Intersection Over Union
    • Non-max Suppression
    • Anchor Boxes
    • YOLO Algorithm
    • Region Proposals
    • Semantic Segmentation with U-Net
    • Transpose Convolutions
    • U-Net Architecture Intuition
    • U-Net Architecture
  • Special Applications: Face recognition & Neural Style Transfer
    • What is Face Recognition?
    • One Shot Learning
    • Siamese Network
    • Triplet Loss
    • Face Verification and Binary Classification
    • What is Neural Style Transfer?
    • What are deep ConvNets learning?
    • Cost Function
    • Content Cost Function
    • Style Cost Function
    • 1D and 3D Generalizations

Sequence Modeling: Recurrent and Recursive Nets

  • Recurrent Neural Networks
    • Why Sequence Models?
    • Notation
    • Recurrent Neural Network Model
    • Backpropagation Through Time
    • Different Types of RNNs
    • Language Model and Sequence Generation
    • Sampling Novel Sequences
    • Vanishing Gradients with RNNs
    • Gated Recurrent Unit (GRU)
    • Long Short Term Memory (LSTM)
    • Bidirectional RNN
    • Deep RNNs
  • Natural Language Processing & Word Embeddings
    • Word Representation
    • Using Word Embeddings
    • Properties of Word Embeddings
    • Embedding Matrix
    • Learning Word Embeddings
    • Word2Vec
    • Negative Sampling
    • GloVe Word Vectors
    • Sentiment Classification
    • Debiasing Word Embeddings
  • Sequence Models & Attention Mechanism
    • Basic Models
    • Picking the Most Likely Sentence
    • Beam Search
    • Refinements to Beam Search
    • Error Analysis in Beam Search
    • Bleu Score
    • Attention Model Intuition
    • Attention Model
    • Speech Recognition
    • Trigger Word Detection
  • Transformer Network
    • Transformer Network Intuition
    • Self-Attention
    • Multi-Head Attention
    • Transformer Network
    • Conclusion and Thank You!

Understanding Deep Learning