5.1-MNIST_KNN.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "authorship_tag": "ABX9TyNukBjvPJlV5nPSBJ6f3mJX"
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "# MNist com KNN\n",
        "\n",
        "Neste experimento foram exploradas configurações de hiper-parâmetros do algoritmo KNN, variando a quantidade de vizinhos mais próximos entre 3,5,7 e 9 e também os pesos possíveis entre 'uniform' e 'distance'. Mantive apenas a métrica de distância Euclidiana, porque a distância Manhattan demorou demais para executar e não apresentou nenhum ganho que justificasse a diferença de tempo de processamento, saltando de 20 segundos para 12 minutos. \n",
        "\n",
        "Para garantir uma reproducibilidade entre este experimento e os demais, o conjunto de dados foi separado em 10 pastas com a técnica de Cross Validation utilizando o estado randômico 42.\n",
        "\n",
        "A combinação dos hiper-parâmetros foi feita utilizando Grid Search, que cruza todas as opções dos dicionários e responde qual é a melhor configuração encontrada. Estas parametrizações são utilizadas para criar um modelo final, que é serializado em um objeto Pickle."
      ],
      "metadata": {
        "id": "ZVL0PvW2Aj3U"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "id": "asOy0NdvuWcw"
      },
      "outputs": [],
      "source": [
        "from sklearn.datasets import fetch_openml\n",
        "\n",
        "import joblib\n",
        "from sklearn.metrics import confusion_matrix,ConfusionMatrixDisplay\n",
        "from sklearn.model_selection import GridSearchCV\n",
        "from sklearn.model_selection import StratifiedKFold\n",
        "\n",
        "from sklearn.neighbors import KNeighborsClassifier\n",
        "\n",
        "import time"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "mnist = fetch_openml('mnist_784')"
      ],
      "metadata": {
        "id": "wyTre3HEu1NR"
      },
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "X = mnist[\"data\"]\n",
        "y = mnist[\"target\"]\n",
        "\n",
        "print(X.shape)\n",
        "print(y.shape)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "VdcD50-5u28u",
        "outputId": "19b21969-ec7f-4c36-a185-4e8a7606bf01"
      },
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(70000, 784)\n",
            "(70000,)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "O objeto $param_grid$ recebe a combinação de todos os hiper-parâmetros que pretendo comparar, como a quantidade de vizinhos próximos e o peso.\n",
        "\n",
        "Também estou definindo a segmentação dos dados em 10 pastas e um random_state de 42 para permitir reproducibilidade ao executar este experimento novamente."
      ],
      "metadata": {
        "id": "cuwBdrblHtp0"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "param_grid = [{'n_neighbors':[3,5,7,9],\n",
        "               'weights':['uniform', 'distance'],\n",
        "               'metric':['euclidean']}]\n",
        "\n",
        "kfolds = StratifiedKFold(n_splits=10, random_state=42, shuffle=True)"
      ],
      "metadata": {
        "id": "TjUlNFVZu7zr"
      },
      "execution_count": 4,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "knn = KNeighborsClassifier()\n",
        "\n",
        "inicio = time.time()\n",
        "grid_search_knn = GridSearchCV(knn, param_grid, cv=kfolds, verbose=3)\n",
        "resultado_modelo = grid_search_knn.fit(X,y)\n",
        "termino = time.time()\n",
        "print(\"--- %s segundos para treinar o modelo ---\" % (termino - inicio))\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "v_dsRIHYw1rc",
        "outputId": "942f575c-130e-44be-cd93-0f67e0690e66"
      },
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Fitting 10 folds for each of 8 candidates, totalling 80 fits\n",
            "[CV 1/10] END metric=euclidean, n_neighbors=3, weights=uniform;, score=0.974 total time=  20.5s\n",
            "[CV 2/10] END metric=euclidean, n_neighbors=3, weights=uniform;, score=0.976 total time=  20.0s\n",
            "[CV 3/10] END metric=euclidean, n_neighbors=3, weights=uniform;, score=0.971 total time=  20.0s\n",
            "[CV 4/10] END metric=euclidean, n_neighbors=3, weights=uniform;, score=0.973 total time=  19.9s\n",
            "[CV 5/10] END metric=euclidean, n_neighbors=3, weights=uniform;, score=0.973 total time=  19.9s\n",
            "[CV 6/10] END metric=euclidean, n_neighbors=3, weights=uniform;, score=0.971 total time=  19.9s\n",
            "[CV 7/10] END metric=euclidean, n_neighbors=3, weights=uniform;, score=0.975 total time=  19.8s\n",
            "[CV 8/10] END metric=euclidean, n_neighbors=3, weights=uniform;, score=0.976 total time=  19.9s\n",
            "[CV 9/10] END metric=euclidean, n_neighbors=3, weights=uniform;, score=0.973 total time=  19.8s\n",
            "[CV 10/10] END metric=euclidean, n_neighbors=3, weights=uniform;, score=0.976 total time=  21.2s\n",
            "[CV 1/10] END metric=euclidean, n_neighbors=3, weights=distance;, score=0.975 total time=  19.7s\n",
            "[CV 2/10] END metric=euclidean, n_neighbors=3, weights=distance;, score=0.977 total time=  19.8s\n",
            "[CV 3/10] END metric=euclidean, n_neighbors=3, weights=distance;, score=0.973 total time=  23.5s\n",
            "[CV 4/10] END metric=euclidean, n_neighbors=3, weights=distance;, score=0.974 total time=  19.7s\n",
            "[CV 5/10] END metric=euclidean, n_neighbors=3, weights=distance;, score=0.974 total time=  19.6s\n",
            "[CV 6/10] END metric=euclidean, n_neighbors=3, weights=distance;, score=0.971 total time=  19.7s\n",
            "[CV 7/10] END metric=euclidean, n_neighbors=3, weights=distance;, score=0.975 total time=  19.7s\n",
            "[CV 8/10] END metric=euclidean, n_neighbors=3, weights=distance;, score=0.976 total time=  19.7s\n",
            "[CV 9/10] END metric=euclidean, n_neighbors=3, weights=distance;, score=0.974 total time=  19.6s\n",
            "[CV 10/10] END metric=euclidean, n_neighbors=3, weights=distance;, score=0.976 total time=  19.6s\n",
            "[CV 1/10] END metric=euclidean, n_neighbors=5, weights=uniform;, score=0.970 total time=  24.1s\n",
            "[CV 2/10] END metric=euclidean, n_neighbors=5, weights=uniform;, score=0.972 total time=  23.3s\n",
            "[CV 3/10] END metric=euclidean, n_neighbors=5, weights=uniform;, score=0.972 total time=  23.3s\n",
            "[CV 4/10] END metric=euclidean, n_neighbors=5, weights=uniform;, score=0.972 total time=  22.7s\n",
            "[CV 5/10] END metric=euclidean, n_neighbors=5, weights=uniform;, score=0.972 total time=  23.0s\n",
            "[CV 6/10] END metric=euclidean, n_neighbors=5, weights=uniform;, score=0.970 total time=  23.2s\n",
            "[CV 7/10] END metric=euclidean, n_neighbors=5, weights=uniform;, score=0.974 total time=  22.9s\n",
            "[CV 8/10] END metric=euclidean, n_neighbors=5, weights=uniform;, score=0.974 total time=  23.0s\n",
            "[CV 9/10] END metric=euclidean, n_neighbors=5, weights=uniform;, score=0.972 total time=  22.9s\n",
            "[CV 10/10] END metric=euclidean, n_neighbors=5, weights=uniform;, score=0.974 total time=  22.8s\n",
            "[CV 1/10] END metric=euclidean, n_neighbors=5, weights=distance;, score=0.972 total time=  23.9s\n",
            "[CV 2/10] END metric=euclidean, n_neighbors=5, weights=distance;, score=0.974 total time=  23.2s\n",
            "[CV 3/10] END metric=euclidean, n_neighbors=5, weights=distance;, score=0.974 total time=  23.1s\n",
            "[CV 4/10] END metric=euclidean, n_neighbors=5, weights=distance;, score=0.974 total time=  22.6s\n",
            "[CV 5/10] END metric=euclidean, n_neighbors=5, weights=distance;, score=0.973 total time=  22.8s\n",
            "[CV 6/10] END metric=euclidean, n_neighbors=5, weights=distance;, score=0.972 total time=  23.4s\n",
            "[CV 7/10] END metric=euclidean, n_neighbors=5, weights=distance;, score=0.975 total time=  22.7s\n",
            "[CV 8/10] END metric=euclidean, n_neighbors=5, weights=distance;, score=0.975 total time=  24.2s\n",
            "[CV 9/10] END metric=euclidean, n_neighbors=5, weights=distance;, score=0.973 total time=  24.6s\n",
            "[CV 10/10] END metric=euclidean, n_neighbors=5, weights=distance;, score=0.975 total time=  27.7s\n",
            "[CV 1/10] END metric=euclidean, n_neighbors=7, weights=uniform;, score=0.968 total time=  25.6s\n",
            "[CV 2/10] END metric=euclidean, n_neighbors=7, weights=uniform;, score=0.973 total time=  24.4s\n",
            "[CV 3/10] END metric=euclidean, n_neighbors=7, weights=uniform;, score=0.971 total time=  23.8s\n",
            "[CV 4/10] END metric=euclidean, n_neighbors=7, weights=uniform;, score=0.973 total time=  23.5s\n",
            "[CV 5/10] END metric=euclidean, n_neighbors=7, weights=uniform;, score=0.971 total time=  23.7s\n",
            "[CV 6/10] END metric=euclidean, n_neighbors=7, weights=uniform;, score=0.970 total time=  24.1s\n",
            "[CV 7/10] END metric=euclidean, n_neighbors=7, weights=uniform;, score=0.974 total time=  24.6s\n",
            "[CV 8/10] END metric=euclidean, n_neighbors=7, weights=uniform;, score=0.973 total time=  24.3s\n",
            "[CV 9/10] END metric=euclidean, n_neighbors=7, weights=uniform;, score=0.971 total time=  24.1s\n",
            "[CV 10/10] END metric=euclidean, n_neighbors=7, weights=uniform;, score=0.973 total time=  24.5s\n",
            "[CV 1/10] END metric=euclidean, n_neighbors=7, weights=distance;, score=0.969 total time=  25.0s\n",
            "[CV 2/10] END metric=euclidean, n_neighbors=7, weights=distance;, score=0.974 total time=  24.3s\n",
            "[CV 3/10] END metric=euclidean, n_neighbors=7, weights=distance;, score=0.972 total time=  24.4s\n",
            "[CV 4/10] END metric=euclidean, n_neighbors=7, weights=distance;, score=0.974 total time=  23.2s\n",
            "[CV 5/10] END metric=euclidean, n_neighbors=7, weights=distance;, score=0.972 total time=  23.2s\n",
            "[CV 6/10] END metric=euclidean, n_neighbors=7, weights=distance;, score=0.971 total time=  23.5s\n",
            "[CV 7/10] END metric=euclidean, n_neighbors=7, weights=distance;, score=0.974 total time=  23.0s\n",
            "[CV 8/10] END metric=euclidean, n_neighbors=7, weights=distance;, score=0.973 total time=  23.2s\n",
            "[CV 9/10] END metric=euclidean, n_neighbors=7, weights=distance;, score=0.972 total time=  24.0s\n",
            "[CV 10/10] END metric=euclidean, n_neighbors=7, weights=distance;, score=0.974 total time=  22.9s\n",
            "[CV 1/10] END metric=euclidean, n_neighbors=9, weights=uniform;, score=0.967 total time=  23.5s\n",
            "[CV 2/10] END metric=euclidean, n_neighbors=9, weights=uniform;, score=0.972 total time=  23.6s\n",
            "[CV 3/10] END metric=euclidean, n_neighbors=9, weights=uniform;, score=0.970 total time=  23.4s\n",
            "[CV 4/10] END metric=euclidean, n_neighbors=9, weights=uniform;, score=0.972 total time=  23.1s\n",
            "[CV 5/10] END metric=euclidean, n_neighbors=9, weights=uniform;, score=0.973 total time=  23.4s\n",
            "[CV 6/10] END metric=euclidean, n_neighbors=9, weights=uniform;, score=0.967 total time=  26.0s\n",
            "[CV 7/10] END metric=euclidean, n_neighbors=9, weights=uniform;, score=0.972 total time=  24.1s\n",
            "[CV 8/10] END metric=euclidean, n_neighbors=9, weights=uniform;, score=0.972 total time=  23.6s\n",
            "[CV 9/10] END metric=euclidean, n_neighbors=9, weights=uniform;, score=0.970 total time=  23.5s\n",
            "[CV 10/10] END metric=euclidean, n_neighbors=9, weights=uniform;, score=0.972 total time=  23.3s\n",
            "[CV 1/10] END metric=euclidean, n_neighbors=9, weights=distance;, score=0.968 total time=  23.6s\n",
            "[CV 2/10] END metric=euclidean, n_neighbors=9, weights=distance;, score=0.973 total time=  23.6s\n",
            "[CV 3/10] END metric=euclidean, n_neighbors=9, weights=distance;, score=0.971 total time=  23.6s\n",
            "[CV 4/10] END metric=euclidean, n_neighbors=9, weights=distance;, score=0.972 total time=  23.2s\n",
            "[CV 5/10] END metric=euclidean, n_neighbors=9, weights=distance;, score=0.973 total time=  24.1s\n",
            "[CV 6/10] END metric=euclidean, n_neighbors=9, weights=distance;, score=0.968 total time=  23.5s\n",
            "[CV 7/10] END metric=euclidean, n_neighbors=9, weights=distance;, score=0.973 total time=  23.3s\n",
            "[CV 8/10] END metric=euclidean, n_neighbors=9, weights=distance;, score=0.972 total time=  23.4s\n",
            "[CV 9/10] END metric=euclidean, n_neighbors=9, weights=distance;, score=0.971 total time=  23.3s\n",
            "[CV 10/10] END metric=euclidean, n_neighbors=9, weights=distance;, score=0.973 total time=  23.2s\n",
            "--- 1822.6732985973358 segundos para treinar o modelo ---\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Dentro do objeto $Grid\\_Search\\_Knn$ é possível recuperar qual foi o melhor conjunto de parâmetros, qual é o melhor *Estimator* e a pontuação que esta combinação de valores alcançou."
      ],
      "metadata": {
        "id": "K11x9HAsILo2"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "print(\"Melhores Parâmetros\")\n",
        "print(grid_search_knn.best_params_)\n",
        "print(\"**************\")\n",
        "print(\"Melhores Estimadores\")\n",
        "print(grid_search_knn.best_estimator_)\n",
        "print(\"**************\")\n",
        "print(\"Melhores Pontuações\")\n",
        "print(grid_search_knn.best_score_)\n",
        "print(\"**************\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "t_wJCbWN2tRT",
        "outputId": "84c3895f-37e5-4024-f70d-a52fd7367709"
      },
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Melhores Parâmetros\n",
            "{'metric': 'euclidean', 'n_neighbors': 3, 'weights': 'distance'}\n",
            "**************\n",
            "Melhores Estimadores\n",
            "KNeighborsClassifier(metric='euclidean', n_neighbors=3, weights='distance')\n",
            "**************\n",
            "Melhores Pontuações\n",
            "0.9744714285714284\n",
            "**************\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Por fim, é possível serializar toda essa configuração do treino do modelo em um objeto único. Neste caso, estou usando o Pickle que é do próprio pacote Sklearn.\n"
      ],
      "metadata": {
        "id": "rIVg4aIFI_2M"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "joblib.dump(grid_search_knn, \"modelo_knn_mnist.pkl\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "sTi7L-b-EiK7",
        "outputId": "920cd0b6-1712-4923-c11f-bc6d82e23dc7"
      },
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['modelo_knn_mnist.pkl']"
            ]
          },
          "metadata": {},
          "execution_count": 14
        }
      ]
    }
  ]
}