RNN_Assignment.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU",
    "gpuClass": "standard"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/imukoki/NLP-FELLOWSHIP/blob/Week-7/RNN_Assignment.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# RNN Model Assignment\n",
        "The aim of this assignment is to make sure you understand the RNN model famility. We have covered RNN, LSTM and GRU. You task is \n",
        "1. to write code for training any of the RNNs ML model that gives the highest accuracy\n",
        "\n",
        "\n",
        "## This we will consider\n",
        "1. The code works\n",
        "2. The hyperparameter used for fine tuning - epoch only is not enough\n",
        "3. The highest accuracy\n",
        "4. Bonus points for being creative with preprocessing, tokenization and creation of vectors\n",
        "\n",
        "# Submissions\n",
        "1. Notebook with code\n",
        "2. 1 models\n",
        "\n",
        "# Deadline\n",
        "Monday 5th"
      ],
      "metadata": {
        "id": "dBTrcNR8o8U0"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from google.colab import drive\n",
        "drive.mount('/content/gdrive')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "NoExJdmVuBCG",
        "outputId": "1199b7c1-4b58-45cc-aeed-d7c5f9f923f0"
      },
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Drive already mounted at /content/gdrive; to attempt to forcibly remount, call drive.mount(\"/content/gdrive\", force_remount=True).\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import os\n",
        "os.chdir(\"/content/gdrive/MyDrive/NLP-FELLOWSHIP/Week 7/\")\n",
        "!ls"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "MACDxYBzuRMP",
        "outputId": "099768e2-76f6-4183-e9ee-b261873887cc"
      },
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "'1_Sequence_to_Sequence_Learning_with_Neural_Networks (1).ipynb'\n",
            " amazon_reviews.csv\n",
            " amazon_reviews.gsheet\n",
            " anna.txt\n",
            "'Copy of Character_Level_RNN_Solution.ipynb'\n",
            "'NLPNN (2).ipynb'\n",
            " RNN_Assignment.ipynb\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "The dataset that was used for this assignment was found on Kaggle, the name of the dataset is [amazon reviews](https://www.kaggle.com/discussions/general/338657)"
      ],
      "metadata": {
        "id": "rAgDqm5bfYzx"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import pandas as pd\n",
        "\n",
        "full_dataset = pd.read_csv('amazon_reviews.csv')\n",
        "full_dataset.head()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 548
        },
        "id": "Cw7dAzUNyCOu",
        "outputId": "28b75167-125e-48a9-b016-ae6b1048de10"
      },
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   Unnamed: 0  reviewerName  overall  \\\n",
              "0           0           NaN      4.0   \n",
              "1           1          0mie      5.0   \n",
              "2           2           1K3      4.0   \n",
              "3           3           1m2      5.0   \n",
              "4           4  2&amp;1/2Men      5.0   \n",
              "\n",
              "                                          reviewText  reviewTime  day_diff  \\\n",
              "0                                         No issues.  2014-07-23       138   \n",
              "1  Purchased this for my device, it worked as adv...  2013-10-25       409   \n",
              "2  it works as expected. I should have sprung for...  2012-12-23       715   \n",
              "3  This think has worked out great.Had a diff. br...  2013-11-21       382   \n",
              "4  Bought it with Retail Packaging, arrived legit...  2013-07-13       513   \n",
              "\n",
              "   helpful_yes  helpful_no  total_vote  score_pos_neg_diff  \\\n",
              "0            0           0           0                   0   \n",
              "1            0           0           0                   0   \n",
              "2            0           0           0                   0   \n",
              "3            0           0           0                   0   \n",
              "4            0           0           0                   0   \n",
              "\n",
              "   score_average_rating  wilson_lower_bound  \n",
              "0                   0.0                 0.0  \n",
              "1                   0.0                 0.0  \n",
              "2                   0.0                 0.0  \n",
              "3                   0.0                 0.0  \n",
              "4                   0.0                 0.0  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-7d893389-0bec-4ba8-aeb3-1cded6d86404\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Unnamed: 0</th>\n",
              "      <th>reviewerName</th>\n",
              "      <th>overall</th>\n",
              "      <th>reviewText</th>\n",
              "      <th>reviewTime</th>\n",
              "      <th>day_diff</th>\n",
              "      <th>helpful_yes</th>\n",
              "      <th>helpful_no</th>\n",
              "      <th>total_vote</th>\n",
              "      <th>score_pos_neg_diff</th>\n",
              "      <th>score_average_rating</th>\n",
              "      <th>wilson_lower_bound</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>4.0</td>\n",
              "      <td>No issues.</td>\n",
              "      <td>2014-07-23</td>\n",
              "      <td>138</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>0mie</td>\n",
              "      <td>5.0</td>\n",
              "      <td>Purchased this for my device, it worked as adv...</td>\n",
              "      <td>2013-10-25</td>\n",
              "      <td>409</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>2</td>\n",
              "      <td>1K3</td>\n",
              "      <td>4.0</td>\n",
              "      <td>it works as expected. I should have sprung for...</td>\n",
              "      <td>2012-12-23</td>\n",
              "      <td>715</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>3</td>\n",
              "      <td>1m2</td>\n",
              "      <td>5.0</td>\n",
              "      <td>This think has worked out great.Had a diff. br...</td>\n",
              "      <td>2013-11-21</td>\n",
              "      <td>382</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>4</td>\n",
              "      <td>2&amp;amp;1/2Men</td>\n",
              "      <td>5.0</td>\n",
              "      <td>Bought it with Retail Packaging, arrived legit...</td>\n",
              "      <td>2013-07-13</td>\n",
              "      <td>513</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-7d893389-0bec-4ba8-aeb3-1cded6d86404')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-7d893389-0bec-4ba8-aeb3-1cded6d86404 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-7d893389-0bec-4ba8-aeb3-1cded6d86404');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Selecting only the reviewText and overall columns\n",
        "dataset = full_dataset[['reviewText', 'overall']]\n",
        "dataset.head()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "uAKHyhoHyRo-",
        "outputId": "b1f9c098-35b1-4a7c-9bfb-0e068cb9a5a1"
      },
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                          reviewText  overall\n",
              "0                                         No issues.      4.0\n",
              "1  Purchased this for my device, it worked as adv...      5.0\n",
              "2  it works as expected. I should have sprung for...      4.0\n",
              "3  This think has worked out great.Had a diff. br...      5.0\n",
              "4  Bought it with Retail Packaging, arrived legit...      5.0"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-9de13e38-3707-407e-ad80-9bbcfb1541c6\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>reviewText</th>\n",
              "      <th>overall</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>No issues.</td>\n",
              "      <td>4.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Purchased this for my device, it worked as adv...</td>\n",
              "      <td>5.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>it works as expected. I should have sprung for...</td>\n",
              "      <td>4.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>This think has worked out great.Had a diff. br...</td>\n",
              "      <td>5.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Bought it with Retail Packaging, arrived legit...</td>\n",
              "      <td>5.0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-9de13e38-3707-407e-ad80-9bbcfb1541c6')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-9de13e38-3707-407e-ad80-9bbcfb1541c6 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-9de13e38-3707-407e-ad80-9bbcfb1541c6');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "for ind,rating in enumerate(dataset['overall']):\n",
        "  if rating >= 3:\n",
        "    dataset['overall'][ind] = 1\n",
        "  else:\n",
        "    dataset['overall'][ind] = 0\n",
        "dataset.head()\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 380
        },
        "id": "FNw9Vp8czalv",
        "outputId": "b037176e-89d7-4d3d-e7c0-81c4da261347"
      },
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "<ipython-input-5-fa0c26ecc879>:3: SettingWithCopyWarning: \n",
            "A value is trying to be set on a copy of a slice from a DataFrame\n",
            "\n",
            "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
            "  dataset['overall'][ind] = 1\n",
            "<ipython-input-5-fa0c26ecc879>:5: SettingWithCopyWarning: \n",
            "A value is trying to be set on a copy of a slice from a DataFrame\n",
            "\n",
            "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
            "  dataset['overall'][ind] = 0\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                          reviewText  overall\n",
              "0                                         No issues.      1.0\n",
              "1  Purchased this for my device, it worked as adv...      1.0\n",
              "2  it works as expected. I should have sprung for...      1.0\n",
              "3  This think has worked out great.Had a diff. br...      1.0\n",
              "4  Bought it with Retail Packaging, arrived legit...      1.0"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-546bc518-4e43-4785-93e2-688ef719e3b2\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>reviewText</th>\n",
              "      <th>overall</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>No issues.</td>\n",
              "      <td>1.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Purchased this for my device, it worked as adv...</td>\n",
              "      <td>1.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>it works as expected. I should have sprung for...</td>\n",
              "      <td>1.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>This think has worked out great.Had a diff. br...</td>\n",
              "      <td>1.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Bought it with Retail Packaging, arrived legit...</td>\n",
              "      <td>1.0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-546bc518-4e43-4785-93e2-688ef719e3b2')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-546bc518-4e43-4785-93e2-688ef719e3b2 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-546bc518-4e43-4785-93e2-688ef719e3b2');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "print(\"No of positive : \"+str(len(dataset[dataset['overall'] == 1])))\n",
        "print(\"No of negative : \"+str(len(dataset[dataset['overall'] == 0])))\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zVbQERgZ0yRd",
        "outputId": "6146812b-1816-4133-805e-2101d8f6a520"
      },
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "No of positive : 4591\n",
            "No of negative : 324\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "split = []\n",
        "for ind in range(len(dataset)):\n",
        "  if ind % 2 == 0:\n",
        "    split.append('train')\n",
        "  else:\n",
        "    split.append('test')\n",
        "\n",
        "dataset['set'] = split\n",
        "dataset.head()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 310
        },
        "id": "IlCEVS-81x5i",
        "outputId": "d80a4e62-4c6e-4525-b942-7bf46bda83f7"
      },
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "<ipython-input-7-7df2d4e9b070>:8: SettingWithCopyWarning: \n",
            "A value is trying to be set on a copy of a slice from a DataFrame.\n",
            "Try using .loc[row_indexer,col_indexer] = value instead\n",
            "\n",
            "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
            "  dataset['set'] = split\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                          reviewText  overall    set\n",
              "0                                         No issues.      1.0  train\n",
              "1  Purchased this for my device, it worked as adv...      1.0   test\n",
              "2  it works as expected. I should have sprung for...      1.0  train\n",
              "3  This think has worked out great.Had a diff. br...      1.0   test\n",
              "4  Bought it with Retail Packaging, arrived legit...      1.0  train"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-8de27a27-f83b-4b79-bc7e-14969008073d\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>reviewText</th>\n",
              "      <th>overall</th>\n",
              "      <th>set</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>No issues.</td>\n",
              "      <td>1.0</td>\n",
              "      <td>train</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Purchased this for my device, it worked as adv...</td>\n",
              "      <td>1.0</td>\n",
              "      <td>test</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>it works as expected. I should have sprung for...</td>\n",
              "      <td>1.0</td>\n",
              "      <td>train</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>This think has worked out great.Had a diff. br...</td>\n",
              "      <td>1.0</td>\n",
              "      <td>test</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Bought it with Retail Packaging, arrived legit...</td>\n",
              "      <td>1.0</td>\n",
              "      <td>train</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-8de27a27-f83b-4b79-bc7e-14969008073d')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-8de27a27-f83b-4b79-bc7e-14969008073d button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-8de27a27-f83b-4b79-bc7e-14969008073d');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "print(\"No of positive in train : \"+str(len(dataset[(dataset['overall'] == 1) & (dataset['set'] == 'train')])))\n",
        "print(\"No of negative in train : \"+str(len(dataset[(dataset['overall'] == 0) & (dataset['set'] == 'train')])))\n",
        "print(\"No of positive in test : \"+str(len(dataset[(dataset['overall'] == 1) & (dataset['set'] == 'test')])))\n",
        "print(\"No of negative in test : \"+str(len(dataset[(dataset['overall'] == 0) & (dataset['set'] == 'test')])))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "MsM5lta631vR",
        "outputId": "02308bd2-9631-489b-eb45-e62b267a732b"
      },
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "No of positive in train : 2304\n",
            "No of negative in train : 154\n",
            "No of positive in test : 2287\n",
            "No of negative in test : 170\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "dataset['reviewText'].describe()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "kk92UjCQ4iUV",
        "outputId": "8583c52e-7328-41e1-dc56-98e66ab32553"
      },
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "count           4914\n",
              "unique          4912\n",
              "top       No issues.\n",
              "freq               2\n",
              "Name: reviewText, dtype: object"
            ]
          },
          "metadata": {},
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "dataset = dataset.drop_duplicates(subset=['reviewText'])"
      ],
      "metadata": {
        "id": "bLSr_seD4zIC"
      },
      "execution_count": 10,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "dataset['reviewText'].describe()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "9howf4q648GC",
        "outputId": "6841d6c4-f67f-4437-f539-3421efc305f9"
      },
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "count           4912\n",
              "unique          4912\n",
              "top       No issues.\n",
              "freq               1\n",
              "Name: reviewText, dtype: object"
            ]
          },
          "metadata": {},
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "print(\"No of positive in train : \"+str(len(dataset[(dataset['overall'] == 1) & (dataset['set'] == 'train')])))\n",
        "print(\"No of negative in train : \"+str(len(dataset[(dataset['overall'] == 0) & (dataset['set'] == 'train')])))\n",
        "print(\"No of positive in test : \"+str(len(dataset[(dataset['overall'] == 1) & (dataset['set'] == 'test')])))\n",
        "print(\"No of negative in test : \"+str(len(dataset[(dataset['overall'] == 0) & (dataset['set'] == 'test')])))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "w70aZPUZ5Ew7",
        "outputId": "2a34d12c-4916-4147-d86b-2e348ed36427"
      },
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "No of positive in train : 2303\n",
            "No of negative in train : 154\n",
            "No of positive in test : 2286\n",
            "No of negative in test : 170\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "dataset['overall'] = dataset['overall'].astype('int')\n",
        "dataset['reviewText'] = dataset['reviewText'].astype('str')\n",
        "dataset.rename(columns={'overall':'sentiment','reviewText':'review'}, inplace = True)\n",
        "dataset.info()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6xEMg8dO_Wp0",
        "outputId": "f0265672-3318-4af5-eb56-7569ebc76af6"
      },
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "<class 'pandas.core.frame.DataFrame'>\n",
            "Int64Index: 4913 entries, 0 to 4914\n",
            "Data columns (total 3 columns):\n",
            " #   Column     Non-Null Count  Dtype \n",
            "---  ------     --------------  ----- \n",
            " 0   review     4913 non-null   object\n",
            " 1   sentiment  4913 non-null   int64 \n",
            " 2   set        4913 non-null   object\n",
            "dtypes: int64(1), object(2)\n",
            "memory usage: 153.5+ KB\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "train_dataset = dataset[dataset['set'] == 'train'][['review','sentiment']]\n",
        "test_dataset = dataset[dataset['set'] == 'test'][['review','sentiment']]\n",
        "test_dataset.head()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "-1fX4BXC5NQi",
        "outputId": "b2963e3c-b0d9-4d9e-9e0f-a459e7678f6d"
      },
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                              review  sentiment\n",
              "1  Purchased this for my device, it worked as adv...          1\n",
              "3  This think has worked out great.Had a diff. br...          1\n",
              "5  It's mini storage.  It doesn't do anything els...          1\n",
              "7  It's hard to believe how affordable digital ha...          1\n",
              "9  in my galaxy s4, super fast card, and am total...          1"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-289bd93f-b9e8-4aac-b569-b9a1d2f572d8\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>review</th>\n",
              "      <th>sentiment</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Purchased this for my device, it worked as adv...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>This think has worked out great.Had a diff. br...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>It's mini storage.  It doesn't do anything els...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>It's hard to believe how affordable digital ha...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>in my galaxy s4, super fast card, and am total...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-289bd93f-b9e8-4aac-b569-b9a1d2f572d8')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-289bd93f-b9e8-4aac-b569-b9a1d2f572d8 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-289bd93f-b9e8-4aac-b569-b9a1d2f572d8');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 14
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "len(test_dataset)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ZVDHdE_lAt_e",
        "outputId": "7a2f8f5f-8d45-4eb4-a591-b0d0eac459bb"
      },
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "2456"
            ]
          },
          "metadata": {},
          "execution_count": 15
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import re\n",
        "def preprocessing(texts):\n",
        "  cleaned_text = []\n",
        "  for text in texts:\n",
        "    text = text.lower()\n",
        "    emoji_pattern = re.compile(\"[\"\n",
        "                                u\"\\U0001F600-\\U0001F64F\"  # emoticons\n",
        "                                u\"\\U0001F300-\\U0001F5FF\"  # symbols & pictographs\n",
        "                                u\"\\U0001F680-\\U0001F6FF\"  # transport & map symbols\n",
        "                                u\"\\U0001F1E0-\\U0001F1FF\"  # flags (iOS)\n",
        "                                u\"\\U00002702-\\U000027B0\"\n",
        "                                u\"\\U000024C2-\\U0001F251\"\n",
        "                                \"]+\", flags=re.UNICODE)\n",
        "    url_pattern = re.compile(r'https?://\\S+|www\\.\\S+')\n",
        "    html_pattern = re.compile('<.*?>')\n",
        "    text = emoji_pattern.sub(r'', text)\n",
        "    text = url_pattern.sub(r'', text)\n",
        "    text = html_pattern.sub(r'', text)\n",
        "    text = re.sub(r\"[^\\w\\d'\\s]+\", ' ', text)\n",
        "    cleaned_text.append(text)\n",
        "\n",
        "  return cleaned_text"
      ],
      "metadata": {
        "id": "796IqS_p5nZ6"
      },
      "execution_count": 16,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install torch==1.8.0 torchtext==0.9.0 #compatibility"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "tbmvLdnO6KN7",
        "outputId": "6690bf45-2406-4624-a126-7e1aa34f702f"
      },
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
            "Requirement already satisfied: torch==1.8.0 in /usr/local/lib/python3.8/dist-packages (1.8.0+cu111)\n",
            "Collecting torchtext==0.9.0\n",
            "  Downloading torchtext-0.9.0-cp38-cp38-manylinux1_x86_64.whl (7.0 MB)\n",
            "\u001b[K     |████████████████████████████████| 7.0 MB 7.5 MB/s \n",
            "\u001b[?25hRequirement already satisfied: typing-extensions in /usr/local/lib/python3.8/dist-packages (from torch==1.8.0) (4.1.1)\n",
            "Requirement already satisfied: numpy in /usr/local/lib/python3.8/dist-packages (from torch==1.8.0) (1.21.6)\n",
            "Requirement already satisfied: tqdm in /usr/local/lib/python3.8/dist-packages (from torchtext==0.9.0) (4.64.1)\n",
            "Requirement already satisfied: requests in /usr/local/lib/python3.8/dist-packages (from torchtext==0.9.0) (2.23.0)\n",
            "Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.8/dist-packages (from requests->torchtext==0.9.0) (3.0.4)\n",
            "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.8/dist-packages (from requests->torchtext==0.9.0) (1.24.3)\n",
            "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.8/dist-packages (from requests->torchtext==0.9.0) (2.10)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.8/dist-packages (from requests->torchtext==0.9.0) (2022.9.24)\n",
            "Installing collected packages: torchtext\n",
            "  Attempting uninstall: torchtext\n",
            "    Found existing installation: torchtext 0.13.1\n",
            "    Uninstalling torchtext-0.13.1:\n",
            "      Successfully uninstalled torchtext-0.13.1\n",
            "Successfully installed torchtext-0.9.0\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#!pip install torch==1.8.0+cu111 torchvision==0.9.0+cu111 torchaudio==0.8.0 -f https://download.pytorch.org/whl/torch_stable.html\n"
      ],
      "metadata": {
        "id": "EL0u-2Khai2X"
      },
      "execution_count": 18,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "import torch\n",
        "from torchtext.legacy import data\n",
        "from torchtext.legacy.data import Dataset, Example\n",
        "from torchtext.legacy.data import BucketIterator\n",
        "\n",
        "\n",
        "SEED = 42\n",
        "\n",
        "torch.manual_seed(SEED)\n",
        "torch.backends.cudnn.deterministic = True # Check this\n",
        "max_document_length = 300 #hyperparameter\n",
        "\n",
        "TEXT = data.Field(lower=True, include_lengths=True,  tokenize='spacy',preprocessing=preprocessing,batch_first=True,  fix_length=max_document_length)\n",
        "LABEL = data.Field(sequential=False, use_vocab=False)\n",
        "\n",
        "class DataFrameDataset(Dataset):\n",
        "    def __init__(self, df: pd.DataFrame, fields: list):\n",
        "        super(DataFrameDataset, self).__init__(\n",
        "            [\n",
        "                Example.fromlist(list(r), fields) \n",
        "                for i, r in df.iterrows()\n",
        "            ], \n",
        "            fields\n",
        "        )\n",
        "\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "hgHRjxvm6PVp",
        "outputId": "8bd47108-2b11-4602-8355-f32e50ca9a84"
      },
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.8/dist-packages/torchtext/data/utils.py:123: UserWarning: Spacy model \"en\" could not be loaded, trying \"en_core_web_sm\" instead\n",
            "  warnings.warn(f'Spacy model \"{language}\" could not be loaded, trying \"{OLD_MODEL_SHORTCUTS[language]}\" instead')\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "torch_valid_dataset, torch_test_dataset = DataFrameDataset(\n",
        "    df=test_dataset, \n",
        "    fields=(\n",
        "        ('review', TEXT),\n",
        "        ('sentiment', LABEL)\n",
        "    )\n",
        ").split() #Text to explain training, testing and validation datasets"
      ],
      "metadata": {
        "id": "0HmdPZ3H6sm6"
      },
      "execution_count": 20,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "torch_train_dataset = DataFrameDataset(\n",
        "    df=train_dataset, \n",
        "    fields=(\n",
        "        ('review', TEXT),\n",
        "        ('sentiment', LABEL)\n",
        "    )\n",
        ")"
      ],
      "metadata": {
        "id": "prmoIBcQ6X8d"
      },
      "execution_count": 21,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "max_size = 30000 #hyperparameter\n",
        "TEXT.build_vocab(torch_train_dataset, max_size=max_size,vectors='fasttext.simple.300d')\n",
        "vocab_size = len(TEXT.vocab)\n",
        "\n"
      ],
      "metadata": {
        "id": "csyDujjfESWY"
      },
      "execution_count": 22,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "vocab_size"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "iJ7mtcQcEerb",
        "outputId": "18137f04-3256-4b7f-b78f-a6431677860f"
      },
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "6352"
            ]
          },
          "metadata": {},
          "execution_count": 23
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "print(TEXT.vocab.freqs.most_common(20))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "rP4RM-ELEkV1",
        "outputId": "ae53ee92-b1ab-4e86-ad1a-fe88bae46f06"
      },
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[(' ', 16903), ('i', 4495), ('the', 4270), ('it', 3771), ('and', 3691), ('to', 2939), ('a', 2721), ('my', 2456), ('this', 2246), ('card', 2139), ('for', 1991), ('in', 1649), ('of', 1602), ('is', 1515), ('with', 1505), ('have', 1118), ('on', 1058), ('that', 981), ('was', 845), ('phone', 812)]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "BATCH_SIZE = 56 #hyperparameter\n",
        "\n",
        "#device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\n",
        "\n",
        "train_iterator, valid_iterator, test_iterator = data.BucketIterator.splits(\n",
        "    (torch_train_dataset, torch_valid_dataset, torch_test_dataset), \n",
        "    batch_size = BATCH_SIZE ,\n",
        "    sort_key=lambda x: len(x.review),\n",
        "    sort_within_batch=True)#,\n",
        "    #device = device)"
      ],
      "metadata": {
        "id": "8yLW3QpZEpkB"
      },
      "execution_count": 25,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "import torch.nn as nn\n",
        "import torch.nn.functional as F\n",
        "class RNN(nn.Module):\n",
        "    def __init__(self, input_dim, hidden_dim, layer_dim, output_dim, dropout_prob):\n",
        "        super(RNN, self).__init__()\n",
        "\n",
        "        # Defining the number of layers and the nodes in each layer\n",
        "        self.hidden_dim = hidden_dim\n",
        "        self.layer_dim = layer_dim\n",
        "\n",
        "        # RNN layers\n",
        "        self.rnn = nn.RNN(\n",
        "            input_dim, hidden_dim, layer_dim, batch_first=True, dropout=dropout_prob\n",
        "        )\n",
        "        # Fully connected layer output layer\n",
        "        self.fc = nn.Linear(hidden_dim, output_dim)\n",
        "\n",
        "    def forward(self, x):\n",
        "        x = x.float()\n",
        "        \n",
        "        # Initializing hidden state for first input with zeros\n",
        "        h0 = torch.zeros(self.layer_dim, x.size(0), self.hidden_dim)\n",
        "       \n",
        "\n",
        "        # Forward propagation by passing in the input and hidden state into the model\n",
        "        out, h0 = self.rnn(x, h0)\n",
        "        \n",
        "        # Reshaping the outputs in the shape of (batch_size, seq_length, hidden_size)\n",
        "        # so that it can fit into the fully connected layer\n",
        "        out = out[:, -1, :]\n",
        "\n",
        "        # Convert the final state to our desired output shape (batch_size, output_dim)\n",
        "        out = self.fc(out)\n",
        "        return out"
      ],
      "metadata": {
        "id": "1VuJr0dwE1by"
      },
      "execution_count": 26,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "lr = 1e-4\n",
        "batch_size = 70\n",
        "dropout_keep_prob = 0.5\n",
        "embedding_size = 300\n",
        "max_document_length = 300 # each sentence has until 100 words\n",
        "vocab_size = len(TEXT.vocab)\n",
        "dev_size = 0.8 # split percentage to train\\validation data\n",
        "max_size = 30000 # maximum vocabulary size\n",
        "seed = 42\n",
        "num_classes = 2\n",
        "\n",
        "num_epochs = 3\n",
        "hidden_size = 256 # hyperparameter\n",
        "hidden_size1 = 300\n",
        "hidden_size2 = 128\n",
        "hidden_size3 = 64\n",
        "num_layers = 10\n",
        "\n",
        "to_train = True\n",
        "\n",
        "model = RNN(max_document_length, hidden_size1, num_layers, num_classes,dropout_keep_prob)"
      ],
      "metadata": {
        "id": "UFQ9Ku2yFAdS"
      },
      "execution_count": 30,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "def accuracy(probs, target):\n",
        "  winners = probs.argmax(dim=1)\n",
        "  corrects = (winners == target)\n",
        "  accuracy = corrects.sum().float() / float(target.size(0))\n",
        "  return accuracy"
      ],
      "metadata": {
        "id": "OlZlltcsFK5I"
      },
      "execution_count": 28,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "best_valid_loss = float('inf')\n",
        "optimizer = torch.optim.Adam(model.parameters(), lr=lr)\n",
        "\n",
        "loss_func = nn.CrossEntropyLoss()\n",
        "\n",
        "for epoch in range(num_epochs):\n",
        "  train_epoch_loss = 0\n",
        "  train_epoch_acc = 0\n",
        "  for batch in train_iterator:\n",
        "      optimizer.zero_grad()\n",
        "      # retrieve text and no. of words\n",
        "      text, text_lengths = batch.review\n",
        "\n",
        "      #feedforward\n",
        "      #predictions = model(text).squeeze(1)\n",
        "      predictions = model(text.unsqueeze(1))\n",
        "      \n",
        "      loss = loss_func(predictions, batch.sentiment)\n",
        "\n",
        "      acc = accuracy(predictions, batch.sentiment)\n",
        "\n",
        "      # perform backpropagation\n",
        "      loss.backward()\n",
        "\n",
        "      optimizer.step()\n",
        "\n",
        "      train_epoch_loss += loss.item()\n",
        "      train_epoch_acc += acc.item()\n",
        "\n",
        "  \n",
        "\n",
        "  valid_epoch_loss = 0\n",
        "  valid_epoch_acc = 0\n",
        "\n",
        "  model.eval()\n",
        "\n",
        "  with torch.no_grad():\n",
        "      for batch in valid_iterator:\n",
        "          text, text_lengths = batch.review\n",
        "\n",
        "          #predictions = model(text).squeeze(1)\n",
        "          predictions = model(text.unsqueeze(1))\n",
        "          loss = loss_func(predictions, batch.sentiment)\n",
        "\n",
        "          acc = accuracy(predictions, batch.sentiment)\n",
        "\n",
        "          valid_epoch_loss += loss.item()\n",
        "          valid_epoch_acc += acc.item()\n",
        "\n",
        "   \n",
        "\n",
        "  if valid_epoch_loss < best_valid_loss:\n",
        "            best_valid_loss = valid_epoch_loss\n",
        "            torch.save(model.state_dict(), 'saved_weights'+'_rnn.pt')\n",
        "\n",
        "  print(f'\\tTrain Loss: {train_epoch_loss / len(train_iterator):.3f} | Train Acc: {train_epoch_acc  / len(train_iterator)* 100:.2f}%')\n",
        "  print(f'\\t Val. Loss: {valid_epoch_loss / len(valid_iterator):.3f} |  Val. Acc: {valid_epoch_acc / len(valid_iterator) * 100:.2f}%')\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "vnFN5Cn8FVu-",
        "outputId": "5b0148f9-e3cf-4014-a1cf-2c9029ee2b21"
      },
      "execution_count": 31,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\tTrain Loss: 0.415 | Train Acc: 93.48%\n",
            "\t Val. Loss: 0.243 |  Val. Acc: 93.32%\n",
            "\tTrain Loss: 0.240 | Train Acc: 93.65%\n",
            "\t Val. Loss: 0.239 |  Val. Acc: 93.32%\n",
            "\tTrain Loss: 0.219 | Train Acc: 93.65%\n",
            "\t Val. Loss: 0.231 |  Val. Acc: 93.32%\n"
          ]
        }
      ]
    }
  ]
}