Transformer_Tutorial_2020.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Transformer_Tutorial",
      "provenance": [],
      "authorship_tag": "ABX9TyN5w6wVx4xx8IdHKzqT1d55",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/ludwigwittgenstein2/Research/blob/master/Transformer_Tutorial_2020.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1OsBk84eFNRP"
      },
      "source": [
        "##Machine Learning: \n",
        "\n",
        "###Attention is all you need: \n",
        "\n",
        "Transformer Model from attention is all you need is foundation for cutting edge research in NLP. \n",
        "\n",
        "In this tutorial, I explain to you basics in Transformer Model.\n",
        "\n",
        "\n",
        "\n",
        "### Why do we need Transformer Model? \n",
        "\n",
        "\n",
        "Named Entity Recognition (NER), \n",
        "Sentiment Analysis, \n",
        "Language Modeling, \n",
        "Question Answering\n",
        "\n",
        "\n",
        "\n",
        "Outline\n",
        "\n",
        "##1) \n",
        "##2) \n",
        "##3) \n",
        "##4) \n",
        "##5) Conclusion \n",
        "\n",
        "\n",
        "##Components of Transformer: \n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "A Transformer consists of Encoder and Decoder \n",
        "\n",
        "\n",
        "Encoder: \n",
        "Number of layers is 6\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "Code: \n",
        "\n",
        "\n",
        "\n",
        "Representing the input: \n",
        "\n",
        "We represent the input using one hot vector \n",
        "\n",
        "A one hot vector is a vector in which every element is zero, except one element which is non-zero, a one. \n",
        "\n",
        "\n",
        "We can’t feed in the transformer, just one hot vector, because they are sparse and does not tell us about characteristic of word. We feed word embedding \n",
        "\n",
        "\n",
        "Positional Encoding:\n",
        "\n",
        "\n",
        "Summary of Input \n",
        "\n",
        "1) Word Embedding\n",
        "2) Positional Encoding \n",
        "3) Final Representation \n",
        "\n",
        "\n",
        "Encoder: \n",
        "\n",
        "\n",
        "\n",
        "\n",
        "Decoder: \n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XeW72nVCFMB2"
      },
      "source": [
        ""
      ]
    }
  ]
}