Week7 - Large scale ML.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Large Scale Machine Learning"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So far, we were able to load the entire data in memory and make models. But this might not be possible in many real life situations. \n",
    "\n",
    "We will now look at how \n",
    "* to handle such large-scale data\n",
    "* do incremental preprocessing and learning\n",
    "    * `fit()` vs `partial_fit()`\n",
    "* Combining preprocessing and incremental learning.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Incremental Learning\n",
    "\n",
    "The following estimators implement `partial_fit` method;\n",
    "\n",
    "* Classification:\n",
    "    * `MultinomialNB`\n",
    "    * `BernoulliNB`\n",
    "    * `SGDClassifier`\n",
    "    * `Perceptron`\n",
    "\n",
    "* Regression:\n",
    "    * `SGDRegressor`\n",
    "* Clustering:    \n",
    "    * `MiniBatchKMeans`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## `fit()` vs `partial_fit()`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import SGDClassifier\n",
    "from sklearn.datasets import make_classification\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import confusion_matrix\n",
    "from sklearn.metrics import classification_report"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. Traditional Approach"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "x, y = make_classification(n_samples=50000, n_features=10,\n",
    "                            n_classes=3,\n",
    "                            n_clusters_per_class=1)\n",
    "\n",
    "x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.15)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "clf1 = SGDClassifier(max_iter=1000, tol=0.01)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-1 {color: black;background-color: white;}#sk-container-id-1 pre{padding: 0;}#sk-container-id-1 div.sk-toggleable {background-color: white;}#sk-container-id-1 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-1 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-1 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-1 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-1 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-1 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-1 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-1 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-1 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-1 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-1 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-1 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-1 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-1 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-1 div.sk-item {position: relative;z-index: 1;}#sk-container-id-1 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-1 div.sk-item::before, #sk-container-id-1 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-1 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-1 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-1 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-1 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-1 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-1 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-1 div.sk-label-container {text-align: center;}#sk-container-id-1 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-1 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-1\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>SGDClassifier(tol=0.01)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-1\" type=\"checkbox\" checked><label for=\"sk-estimator-id-1\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">SGDClassifier</label><div class=\"sk-toggleable__content\"><pre>SGDClassifier(tol=0.01)</pre></div></div></div></div></div>"
      ],
      "text/plain": [
       "SGDClassifier(tol=0.01)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf1.fit(x_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training score:  0.8569882352941176\n"
     ]
    }
   ],
   "source": [
    "train_score = clf1.score(x_train, y_train)\n",
    "print(\"Training score: \", train_score)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training score:  0.8558666666666667\n"
     ]
    }
   ],
   "source": [
    "test_score = clf1.score(x_test, y_test)\n",
    "print(\"Training score: \", test_score)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.89      0.87      0.88      2450\n",
      "           1       0.76      0.93      0.84      2503\n",
      "           2       0.95      0.76      0.85      2547\n",
      "\n",
      "    accuracy                           0.86      7500\n",
      "   macro avg       0.87      0.86      0.86      7500\n",
      "weighted avg       0.87      0.86      0.86      7500\n",
      "\n"
     ]
    }
   ],
   "source": [
    "y_pred = clf1.predict(x_test)\n",
    "cr = classification_report(y_test, y_pred)\n",
    "print(cr)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Incremental Approach"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "train_data = np.concatenate((x_train, y_train[:, np.newaxis]), axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.74275776,  0.49457031, -1.29812498, -0.58699559, -1.1399507 ,\n",
       "        -0.22620716, -0.44226899,  0.40878169,  0.12225937, -1.89362435,\n",
       "         0.        ],\n",
       "       [-1.02595548, -0.03550852,  0.13379618,  1.24075471, -0.83882959,\n",
       "        -0.40484922,  0.80618779, -1.06090471,  0.39657459, -1.27431611,\n",
       "         2.        ],\n",
       "       [-1.52458453,  1.39315769,  0.12053616,  2.79144218, -0.11238248,\n",
       "        -0.96563916,  0.92150961, -2.39903613,  0.93287024,  0.29855201,\n",
       "         0.        ],\n",
       "       [-0.85912332,  0.97274051, -0.57611902,  0.43891912,  0.56196722,\n",
       "         0.30308825,  0.82153158, -0.2758012 , -0.19077736, -0.96869707,\n",
       "         1.        ],\n",
       "       [ 0.0626458 ,  0.83597797,  1.7728184 ,  1.3542036 ,  0.41813458,\n",
       "        -0.27372466, -1.5831475 , -1.12042463,  0.30810859, -1.39087474,\n",
       "         2.        ]])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_data[0:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "a = np.asarray(train_data)\n",
    "np.savetxt(\"train_data.csv\", a, delimiter=\",\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "clf2 = SGDClassifier(max_iter=1000, tol=0.01)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "After iter # 1\n",
      "[[ -8.5769008    7.93797752   5.35265501 -34.9807029   -0.80847561\n",
      "   -7.62726534  -3.79382593  25.66527429   2.94401444   0.88333419]\n",
      " [ -4.30405777   1.08537182   4.56458589  50.11388856  -6.09829687\n",
      "   53.60437629   3.8224841  -27.25428201 -35.87554794 -12.43421994]\n",
      " [ -3.65565886  -4.9161556   10.77939477  -8.64574141   5.08815768\n",
      "  -23.45998127 -23.75614054   1.53363298  16.73174614  -4.93573183]]\n",
      "[ -4.41000429 -87.11168151  -1.74028946]\n",
      "After iter # 2\n",
      "[[-4.86704380e+00 -3.13991446e+00  3.66689684e+00 -1.79652051e+01\n",
      "  -1.55220865e+01  3.88722966e+00 -3.99855187e+00  1.49208896e+01\n",
      "  -4.27729317e+00  9.56908264e+00]\n",
      " [-3.81282101e+00  9.55507099e+00 -5.44200696e+00  3.56356990e+01\n",
      "   3.50755056e-02  3.35361864e+01 -3.83024698e+00 -2.04017458e+01\n",
      "  -2.21123050e+01 -3.44839816e+00]\n",
      " [-1.37242640e+00 -6.22909636e+00 -2.28882221e+00  2.41549001e+00\n",
      "   9.76274909e+00 -2.21285418e+01 -1.08001346e+00 -6.82282160e+00\n",
      "   1.66022370e+01 -2.57739003e+00]]\n",
      "[ -4.52742566 -98.35956702 -12.09251929]\n",
      "After iter # 3\n",
      "[[  0.37697828  -4.26792978  -2.51309595 -22.22863801  11.63659362\n",
      "  -15.86285926  -2.78457369  13.85326822  10.04245935  -9.62143001]\n",
      " [  2.68143237   0.93031264   6.82245579  30.65069821 -13.67148257\n",
      "   40.15326608   9.21819655 -15.02678517 -27.40756445   2.31192472]\n",
      " [  2.92904105   2.11570207  -4.36326933  -1.27466263 -13.90578056\n",
      "  -14.35398883   4.15113163  -2.20279118  10.54882588  10.1130167 ]]\n",
      "[-13.57511866 -74.9337221   -8.2692    ]\n",
      "After iter # 4\n",
      "[[ -1.89719973   4.30612123   6.66142911 -12.17076139  -4.08299785\n",
      "    6.46165175  -3.45767929  10.96177937  -5.73744528  -5.13121093]\n",
      " [ -8.46273143  -3.44356671   6.76618566  36.24367627  -2.20653113\n",
      "   25.07497366   4.55645109 -22.7636484  -15.7886526    4.16109296]\n",
      " [ -2.53626192  -1.68471723 -11.14985537   3.09834523   1.54011528\n",
      "  -11.67963734   0.40332891  -5.02762511   8.90426965   7.19846685]]\n",
      "[ -2.26060594 -70.60492298  -3.95080994]\n",
      "After iter # 5\n",
      "[[  3.43024642  -5.15177742   4.88673403 -10.97504582  -1.2220547\n",
      "   -1.11042128  -0.03930252   8.33830311  -0.02775467   4.62971111]\n",
      " [ -0.96185133  -0.76751724  -4.72245693  24.07185481   0.603078\n",
      "   23.5479416    0.47112974 -13.5819757  -15.60021504   6.87172732]\n",
      " [  1.19892456  -4.74131741   3.38473232  -2.57252983   5.63045152\n",
      "  -11.98479486   2.6975365   -0.65929102   8.69067947   3.28736364]]\n",
      "[ -6.29335535 -54.65812024  -4.38227422]\n",
      "After iter # 6\n",
      "[[ -2.11199662   1.89333219  -3.63850493 -11.21685496  -0.28712616\n",
      "    2.57371292  -1.89754212   9.34878468  -2.7793866    0.98535441]\n",
      " [  4.28807985   2.80616586   2.93781593  21.75855739  -2.12370211\n",
      "   26.35310727   0.57320535 -11.14690548 -17.86054157   2.13728593]\n",
      " [ -5.23910431  -2.2217106   -5.44473384   1.29476249  -1.35010479\n",
      "   -9.53035353  -0.47785449  -3.13752538   7.17001734   0.11801898]]\n",
      "[ -6.32255205 -45.50983123  -5.74270642]\n",
      "After iter # 7\n",
      "[[ -0.71592713   1.80253078   3.97199136  -8.54077408   0.95472053\n",
      "   -2.69095675   1.99592477   6.08160354   1.33353276  -1.56709806]\n",
      " [ -1.58915124  -2.33850886   0.79702377  17.3075791   -2.69077443\n",
      "   14.79933254  -0.75725454 -10.24058535  -9.63532332  -2.04517892]\n",
      " [  0.03599041   2.01628557   3.27255805   1.2699617   -2.91381553\n",
      "   -8.6845134    0.35371339  -2.92955843   6.54065335  -0.73905996]]\n",
      "[ -4.75273807 -34.91619869  -1.8487273 ]\n",
      "After iter # 8\n",
      "[[ -0.51806894  -1.88983265  -4.27122182 -10.09536768  -1.77153566\n",
      "    4.36894756   2.28030315   8.87165352  -4.02407597  -0.23457275]\n",
      " [  2.60564052  -2.56076751  -0.5935039   13.63286313   1.14701318\n",
      "   12.46600798  -8.50579167  -7.8860039   -8.18956466   1.49260211]\n",
      " [  1.37698686   4.0181093    0.88596386  -0.14338724  -2.04996116\n",
      "   -8.35516119   0.71602205  -1.75046283   6.18669195  -0.10873586]]\n",
      "[ -1.36273587 -37.16599804  -5.45195644]\n",
      "After iter # 9\n",
      "[[-2.02324148  2.42225423  1.47456515 -7.46425763 -0.44073207 -2.42255322\n",
      "  -1.06785568  5.29927333  1.21795004 -4.19843766]\n",
      " [ 0.45388951 -0.37954412 -3.21600267 14.64744203 -1.2787182  11.24311346\n",
      "  -1.96314319 -8.95233852 -7.20371525  1.65446947]\n",
      " [ 1.4414319  -0.32477115 -1.2597266   1.63918811  1.37137267 -4.16289825\n",
      "   0.82036188 -2.21039249  3.2151873  -1.64593415]]\n",
      "[ -2.65088883 -32.84567473  -3.16381663]\n",
      "After iter # 10\n",
      "[[-0.525171   -2.52469263  1.94271637 -5.80221039 -1.35162498  1.72015816\n",
      "  -0.01449599  4.92258697 -1.72614825  1.76191919]\n",
      " [-1.52787129  2.04398588  0.49291302 12.86575156 -1.18823072 10.38464438\n",
      "  -3.11437329 -7.74989064 -6.70513331 -0.34559163]\n",
      " [ 2.16495074 -5.1049568   1.73172783 -2.60686474  0.28052051 -4.35297525\n",
      "   3.92875158  1.06894881  3.02677069 -1.60676746]]\n",
      "[ -2.69760095 -29.93252881  -3.05694726]\n",
      "After iter # 11\n",
      "[[-2.45925143  3.87025341  1.05766996 -6.81176085 -4.20546462 -0.62337111\n",
      "   1.85138051  5.1899166  -0.06605249  2.03935085]\n",
      " [ 2.5764143  -2.20141447 -0.3072014   9.38592775 -0.31108923 12.85894694\n",
      "  -1.22515847 -4.47599909 -8.81052802 -0.83847963]\n",
      " [ 1.35699407  0.05839697  2.16252276 -1.43226738 -1.03359867 -4.82187294\n",
      "  -2.29394885  0.04552097  3.46572353 -1.12583804]]\n",
      "[ -2.77010938 -26.41293411  -2.12519862]\n",
      "After iter # 12\n",
      "[[ 2.69863941 -2.0061798   2.29888825 -6.67497875 -0.0494035   0.55752992\n",
      "   2.18107473  5.34617207 -0.93142712 -1.39841916]\n",
      " [ 2.60537178  0.51020234  2.07503863 10.79030918 -0.28302422 10.63907479\n",
      "  -1.24977684 -6.06953738 -7.05488424 -1.71869597]\n",
      " [-1.13680408  1.45641332 -0.77740762 -0.28292603 -0.03035725 -3.67073913\n",
      "  -0.34346104 -0.59699153  2.70098584 -0.84633324]]\n",
      "[ -0.49079297 -25.53341077  -2.96631567]\n",
      "After iter # 13\n",
      "[[-1.47737603  1.18365604  3.69973977 -5.38268818  0.32658861 -1.21352719\n",
      "   1.04384534  3.94037859  0.48259177  0.50271895]\n",
      " [ 1.77870225 -1.06784906  1.32049063  6.83405664  0.17129182  7.96976876\n",
      "  -3.68578644 -3.56960914 -5.38173823  0.30738195]\n",
      " [-1.19957991  2.73233145  1.44585611 -2.56587116 -2.19748179 -5.89012913\n",
      "  -0.26427897  0.69419795  4.17020397  0.75620899]]\n",
      "[ -3.49583058 -21.75960646  -0.08142793]\n",
      "After iter # 14\n",
      "[[-0.24171983  1.37811023  1.54018847 -4.9825216  -0.12046661 -0.18738914\n",
      "   0.60438976  3.85608495 -0.24754674 -0.76520535]\n",
      " [ 1.37894085  0.62494899  0.3903367  10.29521914  1.37374214  8.60881929\n",
      "  -0.83303766 -6.13483473 -5.58726081 -0.98633906]\n",
      " [-1.86100913 -0.38081754 -2.48658414  1.80795897 -1.24250221 -2.65604195\n",
      "   0.91469493 -2.00649606  2.11050243 -1.68190803]]\n",
      "[ -3.44215305 -16.22500044  -2.90772633]\n",
      "After iter # 15\n",
      "[[ 1.2209402  -0.28032935  1.81850717 -4.74207495 -0.79657747 -1.43615664\n",
      "  -0.64157158  3.38959127  0.69743684 -1.2715204 ]\n",
      " [ 1.89447777 -0.2699149  -0.89259165  7.96981189  0.41973106  7.68967551\n",
      "  -0.26077708 -4.52055993 -5.08585299 -1.45569473]\n",
      " [ 0.49631975 -0.98576982  0.69624289 -1.60959559 -1.10818899 -4.38268313\n",
      "   1.97210548  0.28215562  3.12617739  0.75171017]]\n",
      "[ -0.23502505 -16.92152553  -1.61794963]\n",
      "After iter # 16\n",
      "[[-0.6895154  -1.00978138 -0.53661553 -4.66660736  0.38209019 -0.26284836\n",
      "   0.05807778  3.59212082 -0.1670624   2.49505461]\n",
      " [-0.68494784  1.62408689  0.37760065  8.68460445 -1.45155754  6.54762315\n",
      "  -2.56697209 -5.33434734 -4.18322974 -0.06562713]\n",
      " [ 3.49734247 -1.53957436 -0.02537121 -0.41798555  0.82203916 -4.57845577\n",
      "  -1.16152362 -0.69369295  3.36384732  0.12813354]]\n",
      "[ -2.70936842 -18.15681438  -2.21354995]\n",
      "After iter # 17\n",
      "[[ 2.29293067  0.25982814 -0.87187838 -4.61388682  0.2386633  -0.66927732\n",
      "  -1.40628654  3.46027103  0.13851467 -1.95518773]\n",
      " [ 0.96943749  1.78551724  1.02906196  6.3210486  -0.62511165  7.06305922\n",
      "  -0.1435843  -3.37041439 -4.74894559 -0.81147289]\n",
      " [ 0.59922227  1.04353586  0.06400147 -0.21273219  0.52309053 -4.11060996\n",
      "   0.93300087 -0.74996608  3.03272553 -1.27293732]]\n",
      "[ -4.38683517 -14.72893453  -1.62479141]\n",
      "After iter # 18\n",
      "[[ 0.61021757  0.12083046  1.6496658  -3.04968541 -0.16933888  2.80832843\n",
      "  -0.76587234  3.01185657 -2.31980266 -0.67332148]\n",
      " [ 1.23092601  0.59799154  0.46579592  6.80650485 -0.55787442  7.34446483\n",
      "   0.96436767 -3.68745566 -4.92002846 -0.94361562]\n",
      " [ 0.92457295 -0.95594099 -0.67000767  0.36433586  0.78968951 -5.47429879\n",
      "  -0.05577738 -1.50542061  4.08907273  1.58326436]]\n",
      "[ -1.10313355 -15.27582769  -2.72799868]\n",
      "After iter # 19\n",
      "[[-1.56795245  0.6534265   1.54697472 -4.38764582  1.52685527 -0.3679755\n",
      "   0.02208386  3.3504511  -0.06743739 -1.23794987]\n",
      " [-2.31289238  0.59823433  1.8316936   8.331294    0.32838077  6.00204788\n",
      "   1.85070329 -5.17957658 -3.8059351  -0.11958786]\n",
      " [ 0.86066028  2.87087676  0.79358263 -1.22255404  1.22613516 -2.12412944\n",
      "   0.74965042  0.48287508  1.48082096  2.03462176]]\n",
      "[ -2.15561126 -14.77622479  -2.68932513]\n",
      "After iter # 20\n",
      "[[ 0.05849655  0.01104899  1.01295211 -4.11078522 -1.17263779 -0.30544775\n",
      "   1.31743111  3.14780065 -0.09234092 -0.5750609 ]\n",
      " [ 0.26434839  1.47916932 -1.20702853  5.21450246 -0.69721442  5.65462476\n",
      "   0.28984394 -2.81874151 -3.79002348 -0.06618428]\n",
      " [ 0.42492752  1.28514807 -0.52029868  0.89621833  1.22254532 -3.65769131\n",
      "  -0.8820115  -1.51653528  2.78278672  0.8668198 ]]\n",
      "[ -0.70997704 -13.32924486  -0.73435681]\n",
      "After iter # 21\n",
      "[[ 0.47999047  1.4843028  -1.99371139 -3.79544283  1.2514418  -1.57486402\n",
      "   0.78960194  2.61811225  0.87377033  0.06213129]\n",
      " [ 0.31538087 -0.25722222 -1.41666911  5.55181189 -0.31489929  4.53840177\n",
      "  -0.10138615 -3.33146314 -2.93584599  0.83717181]\n",
      " [-2.05938988 -0.97315662  1.27496343  0.20057608  0.73524108 -2.41047107\n",
      "   1.47084022 -0.69428396  1.80363617  1.46317863]]\n",
      "[ -3.06136196 -12.368137    -2.57016488]\n",
      "After iter # 22\n",
      "[[ 0.70429881  0.78488133 -2.1728392  -4.17940282  0.83817713 -0.3527975\n",
      "  -0.38823586  3.19092488 -0.06254057 -1.89139777]\n",
      " [-0.14909939  0.46352608  0.63460136  6.11422001 -1.44982152  4.97775148\n",
      "  -0.80509491 -3.67349353 -3.21812096  0.03728004]\n",
      " [-0.75039038  0.53744795 -0.22474753 -0.4261533  -0.79161035 -2.882254\n",
      "   0.19694196 -0.30916504  2.1049798   0.19629173]]\n",
      "[ -1.70877397 -11.05966145  -2.55390855]\n",
      "After iter # 23\n",
      "[[-3.54612071e-01  3.76541189e-01  5.40950645e-01 -2.53492376e+00\n",
      "  -7.80900980e-01 -1.47417003e+00  1.27481665e-01  1.65444768e+00\n",
      "   8.96868881e-01 -1.75405013e-02]\n",
      " [-8.32042928e-01  1.44603211e-01  1.28591749e+00  5.86482714e+00\n",
      "   1.16363140e-02  4.99983902e+00  5.59155203e-03 -3.47346778e+00\n",
      "  -3.25385363e+00  8.80950300e-01]\n",
      " [-2.14506826e-01  1.70624604e+00  1.36411699e-01 -1.24040428e+00\n",
      "   2.46418671e+00 -3.45954672e+00  3.16315799e+00  1.99131742e-01\n",
      "   2.47004187e+00 -1.50405842e-01]]\n",
      "[ -2.53641697 -10.66263736  -2.9217517 ]\n",
      "After iter # 24\n",
      "[[-0.80539434  0.06126273 -0.26965366 -2.43084707 -1.53696938 -0.68450999\n",
      "  -1.07820136  1.74906855  0.31917749  1.69220736]\n",
      " [ 1.23520894 -1.250051    0.06856682  4.98087083 -0.03046694  5.93906593\n",
      "  -1.43191028 -2.57255878 -4.01914596  0.27962306]\n",
      " [ 0.29071246  0.11649021 -2.27117802 -0.42172169  0.07432318 -3.7207067\n",
      "   0.33496104 -0.49955014  2.72728357 -1.12999714]]\n",
      "[ -1.69592224 -12.71405533  -1.67834697]\n",
      "After iter # 25\n",
      "[[-0.49405713  0.30509126 -0.25609727 -1.86953647 -0.13205511 -0.26438891\n",
      "   0.71856416  1.40361008  0.05108108  1.46401472]\n",
      " [-1.3524126   1.69292505  0.26820622  6.56022705 -0.66571037  5.15295893\n",
      "  -1.17540836 -3.98334842 -3.31348709  0.02190171]\n",
      " [-0.12420996  0.49645917 -0.59270934 -0.84561188 -0.23202882 -1.68553715\n",
      "  -0.60563877  0.2857671   1.18471914 -0.2862402 ]]\n",
      "[ -2.07232068 -11.91728896  -3.2240886 ]\n",
      "After iter # 26\n",
      "[[-0.02233659  0.63331885 -0.99256054 -2.21486393 -0.24318463 -0.46592022\n",
      "   0.07017927  1.62883422  0.17378498  0.24681202]\n",
      " [ 0.83596916 -1.86151978 -0.83386691  4.67253223  0.35701025  4.3648328\n",
      "   1.79583375 -2.68229091 -2.87530917 -0.34754833]\n",
      " [ 0.26138779  0.39164669 -1.25239497  1.07330686  0.70284577 -2.05641127\n",
      "   1.03897877 -1.29809573  1.6087041  -1.12073469]]\n",
      "[ -1.71928432 -11.53734098  -0.57082891]\n",
      "After iter # 27\n",
      "[[-1.22025888 -0.75011332  0.33662264 -2.26580028 -0.80597364 -1.14967356\n",
      "   0.78673194  1.51625133  0.67703803  1.39096717]\n",
      " [ 0.89273967 -1.00440988  0.02696988  4.17475779  0.57710754  4.18131918\n",
      "   1.45687937 -2.33378975 -2.77779773  0.41932664]\n",
      " [ 1.07373052  0.67936625  1.66183466  1.43090406  0.79948366 -2.43081027\n",
      "  -1.42251666 -1.66131206  1.91417427  1.2139365 ]]\n",
      "[ -2.07548419 -10.43465186  -2.40616911]\n",
      "After iter # 28\n",
      "[[ 0.61121544  0.01251206  1.07007449 -3.36435098  0.69347902  0.2011279\n",
      "  -0.61993367  2.67679224 -0.41020675  0.6912656 ]\n",
      " [-0.27864644  2.52409312  0.15880059  4.38670816  0.8404602   6.42828052\n",
      "  -0.41321175 -1.99867959 -4.42813883  0.38020661]\n",
      " [-0.52680647 -0.46681935  0.71419404 -0.7950803  -0.26237708 -3.76191664\n",
      "  -0.27794081 -0.21665623  2.7288872  -1.0934425 ]]\n",
      "[-8.85050847e-04 -1.07771605e+01 -6.44404906e-01]\n",
      "After iter # 29\n",
      "[[ 0.08999071 -0.18384893  0.84361435 -2.02538585  0.0929268  -1.14568876\n",
      "  -0.10485395  1.32906186  0.69273381  0.43546098]\n",
      " [ 0.22248157 -0.12708121  0.03964384  5.36437821  0.71413905  4.24870633\n",
      "  -0.57426535 -3.249415   -2.73549275 -0.54017677]\n",
      " [-0.01454516 -1.3175038   0.63880948  0.09975158 -2.4423989  -1.79231176\n",
      "  -0.91644357 -0.47760059  1.33726663  0.63966059]]\n",
      "[ -0.70945396 -10.45387231  -1.3391047 ]\n",
      "After iter # 30\n",
      "[[-8.99147893e-01 -1.20395753e-02 -5.93059579e-01 -2.31834430e+00\n",
      "   1.44939540e+00 -5.08013458e-01 -2.41197952e-01  1.70040366e+00\n",
      "   1.96981384e-01  5.33802300e-01]\n",
      " [ 3.19980015e-04 -1.64374153e-01 -9.25580914e-01  4.60905184e+00\n",
      "  -8.64430613e-01  4.01388093e+00 -3.84155007e-01 -2.71086829e+00\n",
      "  -2.61989975e+00  8.42499975e-01]\n",
      " [ 6.78733489e-01 -2.82061613e-01 -7.86309081e-01 -1.91083433e+00\n",
      "  -6.46429619e-01 -2.10044418e+00  6.25131064e-01  1.02660158e+00\n",
      "   1.40985360e+00 -8.46297950e-01]]\n",
      "[-1.39467865 -9.48634206 -1.65186625]\n",
      "After iter # 31\n",
      "[[ 0.02039343  0.31691014 -0.76444658 -1.48077475 -0.38678725  0.14152797\n",
      "   0.69794672  1.18997117 -0.21986532 -0.55753131]\n",
      " [-0.42474241 -0.09116661  1.26063406  4.99805539  0.3278686   3.86101166\n",
      "   0.71392242 -3.04926788 -2.47632265 -0.18516773]\n",
      " [ 0.38236181  0.15030964  0.65588356  0.71373581  0.37177263 -2.05963443\n",
      "  -1.03278433 -1.01751943  1.58319899  2.08786364]]\n",
      "[-2.33694714 -9.79465809 -1.31933852]\n",
      "After iter # 32\n",
      "[[-0.52981244  0.8644762   0.34984763 -3.79420241  0.01662259  0.7905326\n",
      "  -0.38394667  3.14446526 -0.88077264 -1.6540368 ]\n",
      " [ 0.72936861 -0.55581797  0.31288549  4.16182089  0.53546615  4.69949419\n",
      "  -0.3980461  -2.20815115 -3.16318099 -0.08606771]\n",
      " [-0.05038073 -0.96083046  0.6258824  -1.27823839 -0.5213346  -3.01595035\n",
      "  -0.72185836  0.3276216   2.13804912 -1.39606225]]\n",
      "[ -2.31301699 -10.08723295  -0.10324699]\n",
      "After iter # 33\n",
      "[[-1.20096563  0.65822095  1.25653096 -2.30112663 -0.1125847   0.4065088\n",
      "   0.87824933  1.89081075 -0.48007081 -0.10501517]\n",
      " [ 0.73103068  1.5944499  -0.8472178   4.73595741  0.8208091   4.34772306\n",
      "   0.49669254 -2.73572317 -2.85769663  1.38055321]\n",
      " [ 0.63703333 -1.41004299 -0.77711041 -1.14944676  1.01533076 -2.75016687\n",
      "   0.98512544  0.28611877  1.95088412 -1.30116291]]\n",
      "[-0.52203542 -8.32113369 -1.01702852]\n",
      "After iter # 34\n",
      "[[ 0.99449728 -0.008655    1.02158108 -3.07228335 -0.37784221 -0.51698416\n",
      "  -0.30132004  2.28821538  0.14514415  0.93932726]\n",
      " [-0.21408588 -0.79929509 -0.98305362  3.82824481  0.68656245  4.78674041\n",
      "  -0.98849705 -1.92774235 -3.25377909 -0.37965623]\n",
      " [ 1.03317239  0.91429769  1.09659037  1.12815659 -0.03433219 -1.60226122\n",
      "  -0.71582081 -1.2397603   1.27607321  0.24423473]]\n",
      "[-2.27257861 -8.6205649  -0.44135143]\n",
      "After iter # 35\n",
      "[[-0.15306808 -0.19440655  0.94989716 -2.67472774 -0.15152157 -1.79154252\n",
      "  -0.7069495   1.69306467  1.121448    0.59768255]\n",
      " [ 0.49406118  0.17729787 -0.35906667  4.48842168 -0.3864141   4.83615793\n",
      "  -0.1205716  -2.43318638 -3.2392193  -0.02398412]\n",
      " [ 0.06511368  0.55617201  0.5383354   0.28053886  0.57260498 -3.24991968\n",
      "   0.72321619 -0.94397962  2.43253851 -0.50239498]]\n",
      "[-1.13927411 -8.91586872 -0.7375179 ]\n",
      "After iter # 36\n",
      "[[-0.84698322  0.24962289  0.57531815 -2.11955901  0.63078463 -0.31107767\n",
      "   0.39322189  1.58879595  0.06631746 -0.08605089]\n",
      " [ 0.03583423 -0.14380163  1.04122201  3.15645537  0.60250453  3.7104462\n",
      "  -0.67631899 -1.64213775 -2.50750789 -0.69245816]\n",
      " [-0.62016199  0.06029982  0.77256784 -0.76317423  0.27357144 -2.21502475\n",
      "   0.62520708  0.10323567  1.58388598 -0.51170801]]\n",
      "[ -1.95142375 -11.09305344  -0.73171624]\n",
      "After iter # 37\n",
      "[[-0.46244651 -0.12353793  0.62365971 -2.14439175  0.55262329 -0.36329244\n",
      "  -0.41609983  1.59658241  0.10312355  0.80661767]\n",
      " [-0.72645107  0.41490979  0.1387757   4.08447097 -0.12217423  3.83854952\n",
      "  -0.30872876 -2.3395722  -2.53053752  0.24951014]\n",
      " [-0.69433243 -0.33317192  0.79296084  0.2861562  -0.346008   -3.09576983\n",
      "   0.1339478  -0.91400912  2.31862674 -0.39236353]]\n",
      "[ -1.15417192 -10.00730491  -0.99954741]\n",
      "After iter # 38\n",
      "[[ 0.51989598 -0.23044744  0.93985901 -2.0529148  -0.18588029 -0.11107995\n",
      "   0.1998065   1.58124569 -0.07686948 -0.72300158]\n",
      " [ 0.36050265 -0.01166534 -0.77640914  3.43507758  0.21001549  3.38521118\n",
      "  -1.24305728 -1.93261117 -2.2446343   0.76582451]\n",
      " [ 0.30574394 -0.08050658 -1.57368567 -0.46006987 -0.92171508 -1.94300402\n",
      "  -0.03046378 -0.07324269  1.40561764  0.20746912]]\n",
      "[-1.66084479 -9.46736936 -1.77181528]\n",
      "After iter # 39\n",
      "[[ 0.0911856   0.47713174  0.34121056 -1.97477709 -0.43683325 -0.54279977\n",
      "   0.62043586  1.42387373  0.24944012 -0.3549266 ]\n",
      " [-0.50908199 -0.97823891 -1.40650294  3.78750356  0.15755345  4.30516809\n",
      "   0.14318961 -2.00322823 -2.89971196 -1.0967288 ]\n",
      " [ 0.22620698 -0.93886466  1.00934973  0.02031087 -0.02349412 -2.49173995\n",
      "   0.07314796 -0.57137874  1.84993578  0.19434265]]\n",
      "[-0.16108843 -9.46089804 -1.25240092]\n",
      "After iter # 40\n",
      "[[-0.08183611  0.71016364 -0.86468986 -2.41791743 -0.64388396  0.73315695\n",
      "   0.44841596  2.05499741 -0.73143755 -0.54560932]\n",
      " [ 0.01221816 -0.55136712  0.04354077  3.93742446 -0.14024371  3.87724739\n",
      "  -0.73112106 -2.21590972 -2.57065148  0.62661647]\n",
      " [-0.33796689 -0.46390981  0.45958188  0.21747382  0.81967543 -2.48408183\n",
      "   0.2359183  -0.7239134   1.85955121  0.27929493]]\n",
      "[-1.40558853 -9.69971844 -1.25412156]\n",
      "After iter # 41\n",
      "[[-0.13796983 -0.46269313 -0.79687215 -2.19770205  0.06105591 -0.34165505\n",
      "  -0.06491522  1.64311109  0.08293716 -0.54248935]\n",
      " [ 0.3064559  -1.00654227 -0.08401907  3.51636738 -0.24241542  3.64733799\n",
      "   0.30892486 -1.93776819 -2.43277206 -0.65331268]\n",
      " [ 0.6112985   1.02766283  0.06411802  0.13961331 -0.62444526 -1.26344175\n",
      "   0.72525275 -0.39088245  0.94804607 -0.0680748 ]]\n",
      "[-1.64233016 -9.44804502 -0.29793093]\n",
      "After iter # 42\n",
      "[[ 1.08824464  0.30341278 -0.20491734 -1.82352352 -0.40524735 -1.21682056\n",
      "  -0.79822491  1.15528602  0.76115979 -0.33232959]\n",
      " [-0.08377644 -0.58675522 -0.33679036  3.40592151  0.12281955  3.58085439\n",
      "   0.24725004 -1.86618699 -2.3920234  -0.02440028]\n",
      " [-0.53633729 -0.1778409  -0.18274722 -0.07995083 -0.12600614 -1.78915734\n",
      "   0.047636   -0.33631496  1.32098513 -1.2981209 ]]\n",
      "[-0.71053036 -8.4903485  -1.24919644]\n",
      "After iter # 43\n",
      "[[-0.55216255  0.63492121  1.1197862  -2.53253548 -0.38855225 -0.64354874\n",
      "  -0.34175651  1.83775166  0.28090351  0.59108093]\n",
      " [ 0.40051323 -0.12698999 -0.38506714  4.16521742 -0.02868684  3.23684639\n",
      "   0.53603645 -2.53687977 -2.07793277  0.97840141]\n",
      " [ 0.43443476 -0.68332148 -0.23208694  0.40612865  1.12960901 -1.59329963\n",
      "  -1.11418998 -0.67291518  1.2134096   0.57702272]]\n",
      "[-1.40834992 -8.25658317 -1.24668905]\n"
     ]
    }
   ],
   "source": [
    "chunksize = 1000\n",
    "iter = 1\n",
    "\n",
    "for train_df in pd.read_csv(\"train_data.csv\", chunksize=chunksize, iterator=True):\n",
    "\n",
    "    if iter == 1:\n",
    "        # In the first iteration, we are specifying all possible class labels\n",
    "        x_train_partial = train_df.iloc[:, 0:10]\n",
    "        y_train_partial = train_df.iloc[:, 10]\n",
    "        clf2.partial_fit(x_train_partial, y_train_partial, classes = np.array([0, 1, 2]))\n",
    "\n",
    "    else:\n",
    "        x_train_partial = train_df.iloc[:, 0:10]\n",
    "        y_train_partial = train_df.iloc[:, 10]\n",
    "        clf2.partial_fit(x_train_partial, y_train_partial)\n",
    "\n",
    "    print(\"After iter #\", iter)\n",
    "    print(clf2.coef_)\n",
    "    print(clf2.intercept_)\n",
    "    iter +=1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training score:  0.8129333333333333\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\Python310\\lib\\site-packages\\sklearn\\base.py:450: UserWarning: X does not have valid feature names, but SGDClassifier was fitted with feature names\n",
      "  warnings.warn(\n"
     ]
    }
   ],
   "source": [
    "test_score = clf2.score(x_test, y_test)\n",
    "print(\"Training score: \", test_score)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.82      0.85      0.84      2450\n",
      "           1       0.78      0.81      0.79      2503\n",
      "           2       0.84      0.78      0.81      2547\n",
      "\n",
      "    accuracy                           0.81      7500\n",
      "   macro avg       0.81      0.81      0.81      7500\n",
      "weighted avg       0.81      0.81      0.81      7500\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\Python310\\lib\\site-packages\\sklearn\\base.py:450: UserWarning: X does not have valid feature names, but SGDClassifier was fitted with feature names\n",
      "  warnings.warn(\n"
     ]
    }
   ],
   "source": [
    "y_pred = clf2.predict(x_test)\n",
    "cr = classification_report(y_test, y_pred)\n",
    "print(cr)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Incremental preprocessing example"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### `CountVectorizer` vs `HashingVectorizer`\n",
    "* `CountVectorizer` and `HashingVectorizer` both perform the task of vectorizing text data\n",
    "* `HashingVectorizer`does't store the resulting vocabulary, therefore it can be used to learn from data that doesn't fit into main memory. Each mini batch is vectorized using `HashingVectorizer` so as to guarantee that that the input space of the vvectorizer has the same dimensionality.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "text = ['Russell was raised by his paternal grandparents after his unconventional parents both died young.', \n",
    "        'He was discontented living with his grandparents, but enjoyed four happy years at Winchester College.',\n",
    "        'His academic education came to a sudden end when he was sent down from Balliol College, Oxford, probably because authorities there had suspicions concerning the nature of his relationship with the future poet Lionel Johnson.',\n",
    "        'He always bitterly resented his treatment by Oxford.']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### `CountVectorizer`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "c_vectorizer = CountVectorizer()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_c = c_vectorizer.fit_transform(text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(4, 56)"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_c.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'russell': 41,\n",
       " 'was': 50,\n",
       " 'raised': 38,\n",
       " 'by': 10,\n",
       " 'his': 27,\n",
       " 'paternal': 35,\n",
       " 'grandparents': 23,\n",
       " 'after': 1,\n",
       " 'unconventional': 49,\n",
       " 'parents': 34,\n",
       " 'both': 8,\n",
       " 'died': 14,\n",
       " 'young': 55,\n",
       " 'he': 26,\n",
       " 'discontented': 15,\n",
       " 'living': 30,\n",
       " 'with': 53,\n",
       " 'but': 9,\n",
       " 'enjoyed': 19,\n",
       " 'four': 20,\n",
       " 'happy': 25,\n",
       " 'years': 54,\n",
       " 'at': 3,\n",
       " 'winchester': 52,\n",
       " 'college': 12,\n",
       " 'academic': 0,\n",
       " 'education': 17,\n",
       " 'came': 11,\n",
       " 'to': 47,\n",
       " 'sudden': 43,\n",
       " 'end': 18,\n",
       " 'when': 51,\n",
       " 'sent': 42,\n",
       " 'down': 16,\n",
       " 'from': 21,\n",
       " 'balliol': 5,\n",
       " 'oxford': 33,\n",
       " 'probably': 37,\n",
       " 'because': 6,\n",
       " 'authorities': 4,\n",
       " 'there': 46,\n",
       " 'had': 24,\n",
       " 'suspicions': 44,\n",
       " 'concerning': 13,\n",
       " 'the': 45,\n",
       " 'nature': 31,\n",
       " 'of': 32,\n",
       " 'relationship': 39,\n",
       " 'future': 22,\n",
       " 'poet': 36,\n",
       " 'lionel': 29,\n",
       " 'johnson': 28,\n",
       " 'always': 2,\n",
       " 'bitterly': 7,\n",
       " 'resented': 40,\n",
       " 'treatment': 48}"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "c_vectorizer.vocabulary_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  (0, 41)\t1\n",
      "  (0, 50)\t1\n",
      "  (0, 38)\t1\n",
      "  (0, 10)\t1\n",
      "  (0, 27)\t2\n",
      "  (0, 35)\t1\n",
      "  (0, 23)\t1\n",
      "  (0, 1)\t1\n",
      "  (0, 49)\t1\n",
      "  (0, 34)\t1\n",
      "  (0, 8)\t1\n",
      "  (0, 14)\t1\n",
      "  (0, 55)\t1\n",
      "  (1, 50)\t1\n",
      "  (1, 27)\t1\n",
      "  (1, 23)\t1\n",
      "  (1, 26)\t1\n",
      "  (1, 15)\t1\n",
      "  (1, 30)\t1\n",
      "  (1, 53)\t1\n",
      "  (1, 9)\t1\n",
      "  (1, 19)\t1\n",
      "  (1, 20)\t1\n",
      "  (1, 25)\t1\n",
      "  (1, 54)\t1\n",
      "  :\t:\n",
      "  (2, 5)\t1\n",
      "  (2, 33)\t1\n",
      "  (2, 37)\t1\n",
      "  (2, 6)\t1\n",
      "  (2, 4)\t1\n",
      "  (2, 46)\t1\n",
      "  (2, 24)\t1\n",
      "  (2, 44)\t1\n",
      "  (2, 13)\t1\n",
      "  (2, 45)\t2\n",
      "  (2, 31)\t1\n",
      "  (2, 32)\t1\n",
      "  (2, 39)\t1\n",
      "  (2, 22)\t1\n",
      "  (2, 36)\t1\n",
      "  (2, 29)\t1\n",
      "  (2, 28)\t1\n",
      "  (3, 10)\t1\n",
      "  (3, 27)\t1\n",
      "  (3, 26)\t1\n",
      "  (3, 33)\t1\n",
      "  (3, 2)\t1\n",
      "  (3, 7)\t1\n",
      "  (3, 40)\t1\n",
      "  (3, 48)\t1\n"
     ]
    }
   ],
   "source": [
    "print(X_c)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### `HashingVectorizer`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.feature_extraction.text import HashingVectorizer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "h_vectorizer = HashingVectorizer(n_features=30)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_h = h_vectorizer.fit_transform(text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(4, 30)"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_h.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  (0, 1)\t-0.2672612419124244\n",
      "  (0, 2)\t-0.2672612419124244\n",
      "  (0, 3)\t-0.2672612419124244\n",
      "  (0, 4)\t0.5345224838248488\n",
      "  (0, 10)\t-0.2672612419124244\n",
      "  (0, 11)\t0.2672612419124244\n",
      "  (0, 19)\t0.0\n",
      "  (0, 20)\t0.2672612419124244\n",
      "  (0, 21)\t-0.2672612419124244\n",
      "  (0, 22)\t0.2672612419124244\n",
      "  (0, 26)\t0.2672612419124244\n",
      "  (0, 28)\t0.2672612419124244\n"
     ]
    }
   ],
   "source": [
    "print(X_h[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Combining preprocessing and fiting in incremental learning"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from io import StringIO, BytesIO, TextIOWrapper\n",
    "from zipfile import ZipFile\n",
    "import urllib.request\n",
    "\n",
    "response = urllib.request.urlopen('https://archive.ics.uci.edu/ml/machine-learning-databases/00331/sentiment%20labelled%20sentences.zip')\n",
    "zipfile = ZipFile(BytesIO(response.read()))\n",
    "\n",
    "data = TextIOWrapper(zipfile.open('sentiment labelled sentences/amazon_cells_labelled.txt'), encoding='utf-8')\n",
    "df = pd.read_csv(data, sep = '\\t')\n",
    "df.columns = ['review', 'sentiment']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>review</th>\n",
       "      <th>sentiment</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Good case, Excellent value.</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Great for the jawbone.</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Tied to charger for conversations lasting more...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>The mic is great.</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>I have to jiggle the plug to get it to line up...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                              review  sentiment\n",
       "0                        Good case, Excellent value.          1\n",
       "1                             Great for the jawbone.          1\n",
       "2  Tied to charger for conversations lasting more...          0\n",
       "3                                  The mic is great.          1\n",
       "4  I have to jiggle the plug to get it to line up...          0"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>review</th>\n",
       "      <th>sentiment</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>994</th>\n",
       "      <td>The screen does get smudged easily because it ...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>995</th>\n",
       "      <td>What a piece of junk.. I lose more calls on th...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>996</th>\n",
       "      <td>Item Does Not Match Picture.</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>997</th>\n",
       "      <td>The only thing that disappoint me is the infra...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>998</th>\n",
       "      <td>You can not answer calls with the unit, never ...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                review  sentiment\n",
       "994  The screen does get smudged easily because it ...          0\n",
       "995  What a piece of junk.. I lose more calls on th...          0\n",
       "996                       Item Does Not Match Picture.          0\n",
       "997  The only thing that disappoint me is the infra...          0\n",
       "998  You can not answer calls with the unit, never ...          0"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.tail()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 999 entries, 0 to 998\n",
      "Data columns (total 2 columns):\n",
      " #   Column     Non-Null Count  Dtype \n",
      "---  ------     --------------  ----- \n",
      " 0   review     999 non-null    object\n",
      " 1   sentiment  999 non-null    int64 \n",
      "dtypes: int64(1), object(1)\n",
      "memory usage: 15.7+ KB\n"
     ]
    }
   ],
   "source": [
    "df.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 0], dtype=int64)"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.loc[:, 'sentiment'].unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "X = df.loc[:, 'review']\n",
    "y = df.loc[:,'sentiment']\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(799,)"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "vectorizer = HashingVectorizer()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "classifier = SGDClassifier(penalty='l2', loss='hinge')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Iteration 1 of `partial_fit()`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train_part1_hashed = vectorizer.fit_transform(X_train[0:400])\n",
    "y_train_part1 = y_train[0:400]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "all_classes = np.unique(df.loc[:, 'sentiment'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-2 {color: black;background-color: white;}#sk-container-id-2 pre{padding: 0;}#sk-container-id-2 div.sk-toggleable {background-color: white;}#sk-container-id-2 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-2 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-2 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-2 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-2 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-2 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-2 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-2 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-2 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-2 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-2 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-2 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-2 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-2 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-2 div.sk-item {position: relative;z-index: 1;}#sk-container-id-2 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-2 div.sk-item::before, #sk-container-id-2 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-2 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-2 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-2 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-2 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-2 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-2 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-2 div.sk-label-container {text-align: center;}#sk-container-id-2 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-2 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-2\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>SGDClassifier()</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-2\" type=\"checkbox\" checked><label for=\"sk-estimator-id-2\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">SGDClassifier</label><div class=\"sk-toggleable__content\"><pre>SGDClassifier()</pre></div></div></div></div></div>"
      ],
      "text/plain": [
       "SGDClassifier()"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "classifier.partial_fit(X_train_part1_hashed, y_train_part1, classes = all_classes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_test_hashed = vectorizer.transform(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test score:  0.745\n"
     ]
    }
   ],
   "source": [
    "test_score = classifier.score(X_test_hashed, y_test)\n",
    "print(\"Test score: \", test_score)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Iteration 2 of `partial_fit()`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train_part2_hashed = vectorizer.fit_transform(X_train[400:])\n",
    "y_train_part2 = y_train[400:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-3 {color: black;background-color: white;}#sk-container-id-3 pre{padding: 0;}#sk-container-id-3 div.sk-toggleable {background-color: white;}#sk-container-id-3 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-3 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-3 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-3 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-3 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-3 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-3 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-3 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-3 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-3 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-3 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-3 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-3 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-3 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-3 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-3 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-3 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-3 div.sk-item {position: relative;z-index: 1;}#sk-container-id-3 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-3 div.sk-item::before, #sk-container-id-3 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-3 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-3 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-3 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-3 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-3 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-3 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-3 div.sk-label-container {text-align: center;}#sk-container-id-3 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-3 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-3\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>SGDClassifier()</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-3\" type=\"checkbox\" checked><label for=\"sk-estimator-id-3\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">SGDClassifier</label><div class=\"sk-toggleable__content\"><pre>SGDClassifier()</pre></div></div></div></div></div>"
      ],
      "text/plain": [
       "SGDClassifier()"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "classifier.partial_fit(X_train_part2_hashed, y_train_part2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test score:  0.705\n"
     ]
    }
   ],
   "source": [
    "test_score = classifier.score(X_test_hashed, y_test)\n",
    "print(\"Test score: \", test_score)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.10.2 64-bit",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.2"
  },
  "orig_nbformat": 4,
  "vscode": {
   "interpreter": {
    "hash": "369f2c481f4da34e4445cda3fffd2e751bd1c4d706f27375911949ba6bb62e1c"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}