5.3-MNIST_MLP.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kbMBkPLBo3yn"
   },
   "source": [
    "#MNist com Multi-Layer Perceptron\n",
    "\n",
    "Neste experimento foram exploradas configurações de hiper-parâmetros do algoritmo Multi-Layer Perceptron, variando a função de ativação (*activation*): entre Tangente Hiperbólica e Relu, o *solver* que é o otimizador do algoritmo entre o *sgd* (Gradiente Descendente Estocástico) e o *adam* (Gradiente Descendente Estocástico otimizado), por fim quantidade de neurônios nas camadas escondidas testando uma unica camada com 100 neurônios, depois são duas camadas com 10 neurônios cada, em seguida duas camadas com 15 e 15, e para finalizar duas camadas com 20 neurônios em cada.\n",
    "\n",
    "Para garantir uma reproducibilidade entre este experimento e os demais, o conjunto de dados foi separado em 10 pastas com a técnica de Cross Validation utilizando o estado randômico 42.\n",
    "\n",
    "A combinação dos hiper-parâmetros foi feita utilizando Grid Search, que cruza todas as opções dos dicionários e responde qual é a melhor configuração encontrada. Estas parametrizações são utilizadas para criar um modelo final, que é serializado em um objeto Pickle."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "id": "7LyrgC7woxqw"
   },
   "outputs": [],
   "source": [
    "from sklearn.datasets import fetch_openml\n",
    "\n",
    "import joblib\n",
    "from sklearn.metrics import confusion_matrix,ConfusionMatrixDisplay\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.model_selection import StratifiedKFold\n",
    "\n",
    "from sklearn.neural_network import MLPClassifier\n",
    "\n",
    "import time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "id": "KgOUxj8IpRxw"
   },
   "outputs": [],
   "source": [
    "mnist = fetch_openml('mnist_784', cache=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "id": "nHbrQU62pR0i"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(70000, 784)\n",
      "(70000,)\n"
     ]
    }
   ],
   "source": [
    "X = mnist[\"data\"]\n",
    "y = mnist[\"target\"]\n",
    "\n",
    "print(X.shape)\n",
    "print(y.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "id": "Bq95OqJupR3a"
   },
   "outputs": [],
   "source": [
    "param_grid = [{'activation': ['tanh', 'relu'],\n",
    "               'solver':['sgd', 'adam'],\n",
    "               'hidden_layer_sizes': [(100),(10,10),(15,15),(20,20)] }]\n",
    "\n",
    "kfolds = StratifiedKFold(n_splits=10, random_state=42, shuffle=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "id": "w8mIAhbKpR5C"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 10 folds for each of 16 candidates, totalling 160 fits\n",
      "[CV 1/10] END activation=tanh, hidden_layer_sizes=100, solver=sgd;, score=0.961 total time=10.6min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\diego nogare\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:702: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (200) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 2/10] END activation=tanh, hidden_layer_sizes=100, solver=sgd;, score=0.963 total time= 9.7min\n",
      "[CV 3/10] END activation=tanh, hidden_layer_sizes=100, solver=sgd;, score=0.964 total time= 6.9min\n",
      "[CV 4/10] END activation=tanh, hidden_layer_sizes=100, solver=sgd;, score=0.959 total time=10.9min\n",
      "[CV 5/10] END activation=tanh, hidden_layer_sizes=100, solver=sgd;, score=0.958 total time= 5.9min\n",
      "[CV 6/10] END activation=tanh, hidden_layer_sizes=100, solver=sgd;, score=0.955 total time= 6.8min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\diego nogare\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:702: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (200) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 7/10] END activation=tanh, hidden_layer_sizes=100, solver=sgd;, score=0.967 total time= 9.4min\n",
      "[CV 8/10] END activation=tanh, hidden_layer_sizes=100, solver=sgd;, score=0.962 total time=10.8min\n",
      "[CV 9/10] END activation=tanh, hidden_layer_sizes=100, solver=sgd;, score=0.957 total time= 6.3min\n",
      "[CV 10/10] END activation=tanh, hidden_layer_sizes=100, solver=sgd;, score=0.962 total time=10.2min\n",
      "[CV 1/10] END activation=tanh, hidden_layer_sizes=100, solver=adam;, score=0.932 total time= 2.2min\n",
      "[CV 2/10] END activation=tanh, hidden_layer_sizes=100, solver=adam;, score=0.951 total time= 8.6min\n",
      "[CV 3/10] END activation=tanh, hidden_layer_sizes=100, solver=adam;, score=0.949 total time= 7.3min\n",
      "[CV 4/10] END activation=tanh, hidden_layer_sizes=100, solver=adam;, score=0.942 total time= 3.8min\n",
      "[CV 5/10] END activation=tanh, hidden_layer_sizes=100, solver=adam;, score=0.943 total time= 6.0min\n",
      "[CV 6/10] END activation=tanh, hidden_layer_sizes=100, solver=adam;, score=0.948 total time= 7.2min\n",
      "[CV 7/10] END activation=tanh, hidden_layer_sizes=100, solver=adam;, score=0.955 total time= 9.2min\n",
      "[CV 8/10] END activation=tanh, hidden_layer_sizes=100, solver=adam;, score=0.955 total time= 6.2min\n",
      "[CV 9/10] END activation=tanh, hidden_layer_sizes=100, solver=adam;, score=0.948 total time= 7.6min\n",
      "[CV 10/10] END activation=tanh, hidden_layer_sizes=100, solver=adam;, score=0.934 total time= 3.2min\n",
      "[CV 1/10] END activation=tanh, hidden_layer_sizes=(10, 10), solver=sgd;, score=0.875 total time= 3.0min\n",
      "[CV 2/10] END activation=tanh, hidden_layer_sizes=(10, 10), solver=sgd;, score=0.863 total time= 3.2min\n",
      "[CV 3/10] END activation=tanh, hidden_layer_sizes=(10, 10), solver=sgd;, score=0.877 total time= 6.4min\n",
      "[CV 4/10] END activation=tanh, hidden_layer_sizes=(10, 10), solver=sgd;, score=0.876 total time= 3.0min\n",
      "[CV 5/10] END activation=tanh, hidden_layer_sizes=(10, 10), solver=sgd;, score=0.874 total time= 6.2min\n",
      "[CV 6/10] END activation=tanh, hidden_layer_sizes=(10, 10), solver=sgd;, score=0.867 total time= 3.1min\n",
      "[CV 7/10] END activation=tanh, hidden_layer_sizes=(10, 10), solver=sgd;, score=0.860 total time= 3.2min\n",
      "[CV 8/10] END activation=tanh, hidden_layer_sizes=(10, 10), solver=sgd;, score=0.880 total time= 2.8min\n",
      "[CV 9/10] END activation=tanh, hidden_layer_sizes=(10, 10), solver=sgd;, score=0.879 total time= 3.1min\n",
      "[CV 10/10] END activation=tanh, hidden_layer_sizes=(10, 10), solver=sgd;, score=0.872 total time= 3.2min\n",
      "[CV 1/10] END activation=tanh, hidden_layer_sizes=(10, 10), solver=adam;, score=0.897 total time= 3.4min\n",
      "[CV 2/10] END activation=tanh, hidden_layer_sizes=(10, 10), solver=adam;, score=0.880 total time= 3.5min\n",
      "[CV 3/10] END activation=tanh, hidden_layer_sizes=(10, 10), solver=adam;, score=0.881 total time= 1.5min\n",
      "[CV 4/10] END activation=tanh, hidden_layer_sizes=(10, 10), solver=adam;, score=0.888 total time= 3.6min\n",
      "[CV 5/10] END activation=tanh, hidden_layer_sizes=(10, 10), solver=adam;, score=0.865 total time= 4.0min\n",
      "[CV 6/10] END activation=tanh, hidden_layer_sizes=(10, 10), solver=adam;, score=0.888 total time= 2.9min\n",
      "[CV 7/10] END activation=tanh, hidden_layer_sizes=(10, 10), solver=adam;, score=0.885 total time= 2.1min\n",
      "[CV 8/10] END activation=tanh, hidden_layer_sizes=(10, 10), solver=adam;, score=0.903 total time= 3.9min\n",
      "[CV 9/10] END activation=tanh, hidden_layer_sizes=(10, 10), solver=adam;, score=0.866 total time= 4.8min\n",
      "[CV 10/10] END activation=tanh, hidden_layer_sizes=(10, 10), solver=adam;, score=0.882 total time= 3.9min\n",
      "[CV 1/10] END activation=tanh, hidden_layer_sizes=(15, 15), solver=sgd;, score=0.894 total time= 3.8min\n",
      "[CV 2/10] END activation=tanh, hidden_layer_sizes=(15, 15), solver=sgd;, score=0.900 total time= 3.3min\n",
      "[CV 3/10] END activation=tanh, hidden_layer_sizes=(15, 15), solver=sgd;, score=0.891 total time= 3.9min\n",
      "[CV 4/10] END activation=tanh, hidden_layer_sizes=(15, 15), solver=sgd;, score=0.894 total time= 5.5min\n",
      "[CV 5/10] END activation=tanh, hidden_layer_sizes=(15, 15), solver=sgd;, score=0.890 total time= 3.4min\n",
      "[CV 6/10] END activation=tanh, hidden_layer_sizes=(15, 15), solver=sgd;, score=0.904 total time= 5.1min\n",
      "[CV 7/10] END activation=tanh, hidden_layer_sizes=(15, 15), solver=sgd;, score=0.909 total time= 6.2min\n",
      "[CV 8/10] END activation=tanh, hidden_layer_sizes=(15, 15), solver=sgd;, score=0.901 total time= 4.1min\n",
      "[CV 9/10] END activation=tanh, hidden_layer_sizes=(15, 15), solver=sgd;, score=0.907 total time= 4.4min\n",
      "[CV 10/10] END activation=tanh, hidden_layer_sizes=(15, 15), solver=sgd;, score=0.903 total time= 5.4min\n",
      "[CV 1/10] END activation=tanh, hidden_layer_sizes=(15, 15), solver=adam;, score=0.902 total time= 3.3min\n",
      "[CV 2/10] END activation=tanh, hidden_layer_sizes=(15, 15), solver=adam;, score=0.912 total time= 4.9min\n",
      "[CV 3/10] END activation=tanh, hidden_layer_sizes=(15, 15), solver=adam;, score=0.911 total time= 4.8min\n",
      "[CV 4/10] END activation=tanh, hidden_layer_sizes=(15, 15), solver=adam;, score=0.896 total time= 3.1min\n",
      "[CV 5/10] END activation=tanh, hidden_layer_sizes=(15, 15), solver=adam;, score=0.915 total time= 6.8min\n",
      "[CV 6/10] END activation=tanh, hidden_layer_sizes=(15, 15), solver=adam;, score=0.916 total time= 5.2min\n",
      "[CV 7/10] END activation=tanh, hidden_layer_sizes=(15, 15), solver=adam;, score=0.912 total time= 4.1min\n",
      "[CV 8/10] END activation=tanh, hidden_layer_sizes=(15, 15), solver=adam;, score=0.913 total time= 4.9min\n",
      "[CV 9/10] END activation=tanh, hidden_layer_sizes=(15, 15), solver=adam;, score=0.921 total time= 8.1min\n",
      "[CV 10/10] END activation=tanh, hidden_layer_sizes=(15, 15), solver=adam;, score=0.907 total time= 3.9min\n",
      "[CV 1/10] END activation=tanh, hidden_layer_sizes=(20, 20), solver=sgd;, score=0.911 total time= 6.2min\n",
      "[CV 2/10] END activation=tanh, hidden_layer_sizes=(20, 20), solver=sgd;, score=0.919 total time= 6.8min\n",
      "[CV 3/10] END activation=tanh, hidden_layer_sizes=(20, 20), solver=sgd;, score=0.907 total time= 3.5min\n",
      "[CV 4/10] END activation=tanh, hidden_layer_sizes=(20, 20), solver=sgd;, score=0.900 total time= 4.1min\n",
      "[CV 5/10] END activation=tanh, hidden_layer_sizes=(20, 20), solver=sgd;, score=0.912 total time= 6.9min\n",
      "[CV 6/10] END activation=tanh, hidden_layer_sizes=(20, 20), solver=sgd;, score=0.906 total time= 3.7min\n",
      "[CV 7/10] END activation=tanh, hidden_layer_sizes=(20, 20), solver=sgd;, score=0.922 total time= 5.2min\n",
      "[CV 8/10] END activation=tanh, hidden_layer_sizes=(20, 20), solver=sgd;, score=0.910 total time= 4.1min\n",
      "[CV 9/10] END activation=tanh, hidden_layer_sizes=(20, 20), solver=sgd;, score=0.918 total time= 4.3min\n",
      "[CV 10/10] END activation=tanh, hidden_layer_sizes=(20, 20), solver=sgd;, score=0.913 total time= 4.2min\n",
      "[CV 1/10] END activation=tanh, hidden_layer_sizes=(20, 20), solver=adam;, score=0.917 total time= 2.3min\n",
      "[CV 2/10] END activation=tanh, hidden_layer_sizes=(20, 20), solver=adam;, score=0.911 total time= 3.0min\n",
      "[CV 3/10] END activation=tanh, hidden_layer_sizes=(20, 20), solver=adam;, score=0.920 total time= 3.4min\n",
      "[CV 4/10] END activation=tanh, hidden_layer_sizes=(20, 20), solver=adam;, score=0.918 total time= 3.0min\n",
      "[CV 5/10] END activation=tanh, hidden_layer_sizes=(20, 20), solver=adam;, score=0.915 total time= 4.4min\n",
      "[CV 6/10] END activation=tanh, hidden_layer_sizes=(20, 20), solver=adam;, score=0.918 total time= 3.9min\n",
      "[CV 7/10] END activation=tanh, hidden_layer_sizes=(20, 20), solver=adam;, score=0.926 total time= 4.9min\n",
      "[CV 8/10] END activation=tanh, hidden_layer_sizes=(20, 20), solver=adam;, score=0.922 total time= 4.3min\n",
      "[CV 9/10] END activation=tanh, hidden_layer_sizes=(20, 20), solver=adam;, score=0.912 total time= 2.8min\n",
      "[CV 10/10] END activation=tanh, hidden_layer_sizes=(20, 20), solver=adam;, score=0.914 total time= 3.6min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\diego nogare\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:702: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (200) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 1/10] END activation=relu, hidden_layer_sizes=100, solver=sgd;, score=0.928 total time=10.5min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\diego nogare\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:702: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (200) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 2/10] END activation=relu, hidden_layer_sizes=100, solver=sgd;, score=0.933 total time= 9.5min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\diego nogare\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:702: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (200) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 3/10] END activation=relu, hidden_layer_sizes=100, solver=sgd;, score=0.938 total time= 9.5min\n",
      "[CV 4/10] END activation=relu, hidden_layer_sizes=100, solver=sgd;, score=0.936 total time= 8.8min\n",
      "[CV 5/10] END activation=relu, hidden_layer_sizes=100, solver=sgd;, score=0.928 total time= 6.8min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\diego nogare\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:702: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (200) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 6/10] END activation=relu, hidden_layer_sizes=100, solver=sgd;, score=0.936 total time=10.3min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\diego nogare\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:702: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (200) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 7/10] END activation=relu, hidden_layer_sizes=100, solver=sgd;, score=0.938 total time= 9.7min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\diego nogare\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:702: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (200) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 8/10] END activation=relu, hidden_layer_sizes=100, solver=sgd;, score=0.910 total time=10.4min\n",
      "[CV 9/10] END activation=relu, hidden_layer_sizes=100, solver=sgd;, score=0.939 total time= 8.9min\n",
      "[CV 10/10] END activation=relu, hidden_layer_sizes=100, solver=sgd;, score=0.937 total time= 8.8min\n",
      "[CV 1/10] END activation=relu, hidden_layer_sizes=100, solver=adam;, score=0.966 total time= 5.5min\n",
      "[CV 2/10] END activation=relu, hidden_layer_sizes=100, solver=adam;, score=0.969 total time= 7.2min\n",
      "[CV 3/10] END activation=relu, hidden_layer_sizes=100, solver=adam;, score=0.965 total time= 3.8min\n",
      "[CV 4/10] END activation=relu, hidden_layer_sizes=100, solver=adam;, score=0.966 total time= 3.3min\n",
      "[CV 5/10] END activation=relu, hidden_layer_sizes=100, solver=adam;, score=0.967 total time= 3.0min\n",
      "[CV 6/10] END activation=relu, hidden_layer_sizes=100, solver=adam;, score=0.960 total time= 4.9min\n",
      "[CV 7/10] END activation=relu, hidden_layer_sizes=100, solver=adam;, score=0.968 total time= 5.0min\n",
      "[CV 8/10] END activation=relu, hidden_layer_sizes=100, solver=adam;, score=0.971 total time= 4.6min\n",
      "[CV 9/10] END activation=relu, hidden_layer_sizes=100, solver=adam;, score=0.966 total time= 5.4min\n",
      "[CV 10/10] END activation=relu, hidden_layer_sizes=100, solver=adam;, score=0.966 total time= 5.3min\n",
      "[CV 1/10] END activation=relu, hidden_layer_sizes=(10, 10), solver=sgd;, score=0.113 total time=  48.3s\n",
      "[CV 2/10] END activation=relu, hidden_layer_sizes=(10, 10), solver=sgd;, score=0.113 total time=  47.2s\n",
      "[CV 3/10] END activation=relu, hidden_layer_sizes=(10, 10), solver=sgd;, score=0.611 total time= 4.5min\n",
      "[CV 4/10] END activation=relu, hidden_layer_sizes=(10, 10), solver=sgd;, score=0.113 total time=  42.5s\n",
      "[CV 5/10] END activation=relu, hidden_layer_sizes=(10, 10), solver=sgd;, score=0.113 total time=  42.2s\n",
      "[CV 6/10] END activation=relu, hidden_layer_sizes=(10, 10), solver=sgd;, score=0.210 total time= 2.7min\n",
      "[CV 7/10] END activation=relu, hidden_layer_sizes=(10, 10), solver=sgd;, score=0.208 total time= 2.8min\n",
      "[CV 8/10] END activation=relu, hidden_layer_sizes=(10, 10), solver=sgd;, score=0.112 total time=  40.5s\n",
      "[CV 9/10] END activation=relu, hidden_layer_sizes=(10, 10), solver=sgd;, score=0.112 total time=  51.2s\n",
      "[CV 10/10] END activation=relu, hidden_layer_sizes=(10, 10), solver=sgd;, score=0.208 total time= 1.9min\n",
      "[CV 1/10] END activation=relu, hidden_layer_sizes=(10, 10), solver=adam;, score=0.874 total time= 4.6min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\diego nogare\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:702: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (200) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 2/10] END activation=relu, hidden_layer_sizes=(10, 10), solver=adam;, score=0.916 total time= 8.8min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\diego nogare\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:702: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (200) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 3/10] END activation=relu, hidden_layer_sizes=(10, 10), solver=adam;, score=0.902 total time= 8.6min\n",
      "[CV 4/10] END activation=relu, hidden_layer_sizes=(10, 10), solver=adam;, score=0.923 total time= 7.1min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\diego nogare\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:702: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (200) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 5/10] END activation=relu, hidden_layer_sizes=(10, 10), solver=adam;, score=0.872 total time= 8.9min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\diego nogare\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:702: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (200) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 6/10] END activation=relu, hidden_layer_sizes=(10, 10), solver=adam;, score=0.890 total time= 8.3min\n",
      "[CV 7/10] END activation=relu, hidden_layer_sizes=(10, 10), solver=adam;, score=0.905 total time= 6.4min\n",
      "[CV 8/10] END activation=relu, hidden_layer_sizes=(10, 10), solver=adam;, score=0.911 total time= 7.0min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\diego nogare\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:702: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (200) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 9/10] END activation=relu, hidden_layer_sizes=(10, 10), solver=adam;, score=0.905 total time= 8.8min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\diego nogare\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:702: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (200) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 10/10] END activation=relu, hidden_layer_sizes=(10, 10), solver=adam;, score=0.871 total time= 8.2min\n",
      "[CV 1/10] END activation=relu, hidden_layer_sizes=(15, 15), solver=sgd;, score=0.113 total time=  49.3s\n",
      "[CV 2/10] END activation=relu, hidden_layer_sizes=(15, 15), solver=sgd;, score=0.820 total time= 5.7min\n",
      "[CV 3/10] END activation=relu, hidden_layer_sizes=(15, 15), solver=sgd;, score=0.288 total time= 1.9min\n",
      "[CV 4/10] END activation=relu, hidden_layer_sizes=(15, 15), solver=sgd;, score=0.750 total time= 4.9min\n",
      "[CV 5/10] END activation=relu, hidden_layer_sizes=(15, 15), solver=sgd;, score=0.113 total time=  49.9s\n",
      "[CV 6/10] END activation=relu, hidden_layer_sizes=(15, 15), solver=sgd;, score=0.113 total time=  49.3s\n",
      "[CV 7/10] END activation=relu, hidden_layer_sizes=(15, 15), solver=sgd;, score=0.304 total time= 3.8min\n",
      "[CV 8/10] END activation=relu, hidden_layer_sizes=(15, 15), solver=sgd;, score=0.112 total time=  50.5s\n",
      "[CV 9/10] END activation=relu, hidden_layer_sizes=(15, 15), solver=sgd;, score=0.112 total time=  44.3s\n",
      "[CV 10/10] END activation=relu, hidden_layer_sizes=(15, 15), solver=sgd;, score=0.112 total time=  51.3s\n",
      "[CV 1/10] END activation=relu, hidden_layer_sizes=(15, 15), solver=adam;, score=0.933 total time= 6.5min\n",
      "[CV 2/10] END activation=relu, hidden_layer_sizes=(15, 15), solver=adam;, score=0.933 total time= 6.9min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\diego nogare\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:702: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (200) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 3/10] END activation=relu, hidden_layer_sizes=(15, 15), solver=adam;, score=0.939 total time= 9.2min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\diego nogare\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:702: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (200) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 4/10] END activation=relu, hidden_layer_sizes=(15, 15), solver=adam;, score=0.923 total time= 9.2min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\diego nogare\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:702: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (200) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 5/10] END activation=relu, hidden_layer_sizes=(15, 15), solver=adam;, score=0.931 total time= 8.6min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\diego nogare\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:702: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (200) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 6/10] END activation=relu, hidden_layer_sizes=(15, 15), solver=adam;, score=0.912 total time= 9.2min\n",
      "[CV 7/10] END activation=relu, hidden_layer_sizes=(15, 15), solver=adam;, score=0.935 total time= 8.1min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\diego nogare\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:702: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (200) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 8/10] END activation=relu, hidden_layer_sizes=(15, 15), solver=adam;, score=0.940 total time= 8.7min\n",
      "[CV 9/10] END activation=relu, hidden_layer_sizes=(15, 15), solver=adam;, score=0.914 total time= 5.6min\n",
      "[CV 10/10] END activation=relu, hidden_layer_sizes=(15, 15), solver=adam;, score=0.930 total time= 7.7min\n",
      "[CV 1/10] END activation=relu, hidden_layer_sizes=(20, 20), solver=sgd;, score=0.827 total time= 6.2min\n",
      "[CV 2/10] END activation=relu, hidden_layer_sizes=(20, 20), solver=sgd;, score=0.113 total time=  38.5s\n",
      "[CV 3/10] END activation=relu, hidden_layer_sizes=(20, 20), solver=sgd;, score=0.879 total time= 7.8min\n",
      "[CV 4/10] END activation=relu, hidden_layer_sizes=(20, 20), solver=sgd;, score=0.841 total time= 7.9min\n",
      "[CV 5/10] END activation=relu, hidden_layer_sizes=(20, 20), solver=sgd;, score=0.382 total time= 5.8min\n",
      "[CV 6/10] END activation=relu, hidden_layer_sizes=(20, 20), solver=sgd;, score=0.653 total time= 4.7min\n",
      "[CV 7/10] END activation=relu, hidden_layer_sizes=(20, 20), solver=sgd;, score=0.873 total time= 6.0min\n",
      "[CV 8/10] END activation=relu, hidden_layer_sizes=(20, 20), solver=sgd;, score=0.526 total time= 3.0min\n",
      "[CV 9/10] END activation=relu, hidden_layer_sizes=(20, 20), solver=sgd;, score=0.112 total time=  48.8s\n",
      "[CV 10/10] END activation=relu, hidden_layer_sizes=(20, 20), solver=sgd;, score=0.905 total time= 6.1min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\diego nogare\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:702: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (200) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 1/10] END activation=relu, hidden_layer_sizes=(20, 20), solver=adam;, score=0.938 total time= 9.4min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\diego nogare\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:702: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (200) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 2/10] END activation=relu, hidden_layer_sizes=(20, 20), solver=adam;, score=0.908 total time=10.1min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\diego nogare\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:702: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (200) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 3/10] END activation=relu, hidden_layer_sizes=(20, 20), solver=adam;, score=0.929 total time= 9.6min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\diego nogare\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:702: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (200) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 4/10] END activation=relu, hidden_layer_sizes=(20, 20), solver=adam;, score=0.932 total time= 9.7min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\diego nogare\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:702: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (200) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 5/10] END activation=relu, hidden_layer_sizes=(20, 20), solver=adam;, score=0.930 total time= 9.2min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\diego nogare\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:702: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (200) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 6/10] END activation=relu, hidden_layer_sizes=(20, 20), solver=adam;, score=0.931 total time= 9.6min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\diego nogare\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:702: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (200) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 7/10] END activation=relu, hidden_layer_sizes=(20, 20), solver=adam;, score=0.939 total time= 9.7min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\diego nogare\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:702: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (200) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 8/10] END activation=relu, hidden_layer_sizes=(20, 20), solver=adam;, score=0.938 total time= 9.8min\n",
      "[CV 9/10] END activation=relu, hidden_layer_sizes=(20, 20), solver=adam;, score=0.946 total time= 6.3min\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\diego nogare\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:702: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (200) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV 10/10] END activation=relu, hidden_layer_sizes=(20, 20), solver=adam;, score=0.944 total time= 9.3min\n",
      "--- 52900.93397808075 segundos para treinar o modelo ---\n"
     ]
    }
   ],
   "source": [
    "mlp = MLPClassifier()\n",
    "\n",
    "inicio = time.time()\n",
    "grid_search_mlp = GridSearchCV(mlp, param_grid, cv=kfolds, verbose=3)\n",
    "resultado_modelo = grid_search_mlp.fit(X,y)\n",
    "termino = time.time()\n",
    "print(\"--- %s segundos para treinar o modelo ---\" % (termino - inicio))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "id": "xLrmWC6fpR7o"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Melhores Parâmetros\n",
      "{'activation': 'relu', 'hidden_layer_sizes': 100, 'solver': 'adam'}\n",
      "**************\n",
      "Melhores Estimadores\n",
      "MLPClassifier(hidden_layer_sizes=100)\n",
      "**************\n",
      "Melhores Pontuações\n",
      "0.9664428571428573\n",
      "**************\n",
      "9\n",
      "**************\n"
     ]
    }
   ],
   "source": [
    "print(\"Melhores Parâmetros\")\n",
    "print(grid_search_mlp.best_params_)\n",
    "print(\"**************\")\n",
    "print(\"Melhores Estimadores\")\n",
    "print(grid_search_mlp.best_estimator_)\n",
    "print(\"**************\")\n",
    "print(\"Melhores Pontuações\")\n",
    "print(grid_search_mlp.best_score_)\n",
    "print(\"**************\")\n",
    "print(grid_search_mlp.best_index_)\n",
    "print(\"**************\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "id": "ki1_vkFOqtE8"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['modelo_mlp_mnist.pkl']"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "joblib.dump(grid_search_mlp, \"modelo_mlp_mnist.pkl\")"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "authorship_tag": "ABX9TyONsIcAsX1tBJr+0qE2uFbM",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}