Data Outcomes using Linear Regression Project IV.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Health Data Outcomes using Linear Regression Project IV\n",
    "\n",
    "**Grupo** <br />\n",
    "Rafael Guimarães <br />\n",
    "Bruno Monteiro   <br />\n",
    "Harrison Santos  <br />\n",
    "\n",
    "\n",
    "**Disciplina:** Métodos Numéricos <br />\n",
    "**Professor:** Paulo Ribeiro"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn import datasets, linear_model, metrics\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.preprocessing import PolynomialFeatures\n",
    "import math, scipy, numpy as np\n",
    "from scipy import linalg\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "O conjuto de dados escolhido para a replicação do metodo de regressão linear [[1]](https://nbviewer.jupyter.org/github/fastai/numerical-linear-algebra/blob/master/nbs/5.%20Health%20Outcomes%20with%20Linear%20Regression.ipynb#5.-Health-Outcomes-with-Linear-Regression) está disponível na documentação do sklearn [[2]](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_breast_cancer.html) referente ao conjunto de dados do cancêr de mama, em Wisconsin. USA. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = datasets.load_breast_cancer()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [],
   "source": [
    "feature_names = ['mean radius', 'mean texture', 'mean perimeter', 'mean area',\n",
    "        'mean smoothness', 'mean compactness', 'mean concavity',\n",
    "        'mean concave points', 'mean symmetry', 'mean fractal dimension',\n",
    "        'radius error', 'texture error', 'perimeter error', 'area error',\n",
    "        'smoothness error', 'compactness error', 'concavity error',\n",
    "        'concave points error', 'symmetry error',\n",
    "        'fractal dimension error', 'worst radius', 'worst texture',\n",
    "        'worst perimeter', 'worst area', 'worst smoothness',\n",
    "        'worst compactness', 'worst concavity', 'worst concave points',\n",
    "        'worst symmetry', 'worst fractal dimension']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [],
   "source": [
    "trn,test,y_trn,y_test = train_test_split(data.data, data.target, test_size=0.2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((455, 30), (114, 30))"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trn.shape, test.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Regressão linear no Scikit Learn\n",
    "\n",
    "Considerando um sistema $$X\\beta = y$$, em que $X$ possui mais linhas que colunas. Isso ocorre quando você tem mais amostras de dados do que variáveis. Queremos encontrar β que minimize: \n",
    "\n",
    "\n",
    "$$∣∣X\\beta − y∣∣_{2}$$ \n",
    "\n",
    "Vamos executar a implementação do sklearn:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "960 µs ± 7.59 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)\n"
     ]
    }
   ],
   "source": [
    "regr = linear_model.LinearRegression()\n",
    "%timeit regr.fit(trn, y_trn)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [],
   "source": [
    "pred = regr.predict(test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A seguir veremos algumas métricas sobre o quão boa é nossa previsão. Veremos a norma quadrática média (L2) e o erro absoluto médio (L1)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [],
   "source": [
    "def regr_metrics(act, pred):\n",
    "    return (math.sqrt(metrics.mean_squared_error(act, pred)), \n",
    "     metrics.mean_absolute_error(act, pred))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(0.2745650835026591, 0.2146580502783168)"
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "regr_metrics(y_test, regr.predict(test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(455, 30)"
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trn.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [],
   "source": [
    "poly = PolynomialFeatures(include_bias=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [],
   "source": [
    "trn_feat = poly.fit_transform(trn)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'mean radius, mean texture, mean perimeter, mean area, mean smoothness, mean compactness, mean concavity, mean concave points, mean symmetry, mean fractal dimension, radius error, texture error, perimeter error, area error, smoothness error, compactness error, concavity error, concave points error, symmetry error, fractal dimension error, worst radius, worst texture, worst perimeter, worst area, worst smoothness, worst compactness, worst concavity, worst concave points, worst symmetry, worst fractal dimension, mean radius^2, mean radius mean texture, mean radius mean perimeter, mean radius mean area, mean radius mean smoothness, mean radius mean compactness, mean radius mean concavity, mean radius mean concave points, mean radius mean symmetry, mean radius mean fractal dimension, mean radius radius error, mean radius texture error, mean radius perimeter error, mean radius area error, mean radius smoothness error, mean radius compactness error, mean radius concavity error, mean radius concave points error, mean radius symmetry error, mean radius fractal dimension error, mean radius worst radius, mean radius worst texture, mean radius worst perimeter, mean radius worst area, mean radius worst smoothness, mean radius worst compactness, mean radius worst concavity, mean radius worst concave points, mean radius worst symmetry, mean radius worst fractal dimension, mean texture^2, mean texture mean perimeter, mean texture mean area, mean texture mean smoothness, mean texture mean compactness, mean texture mean concavity, mean texture mean concave points, mean texture mean symmetry, mean texture mean fractal dimension, mean texture radius error, mean texture texture error, mean texture perimeter error, mean texture area error, mean texture smoothness error, mean texture compactness error, mean texture concavity error, mean texture concave points error, mean texture symmetry error, mean texture fractal dimension error, mean texture worst radius, mean texture worst texture, mean texture worst perimeter, mean texture worst area, mean texture worst smoothness, mean texture worst compactness, mean texture worst concavity, mean texture worst concave points, mean texture worst symmetry, mean texture worst fractal dimension, mean perimeter^2, mean perimeter mean area, mean perimeter mean smoothness, mean perimeter mean compactness, mean perimeter mean concavity, mean perimeter mean concave points, mean perimeter mean symmetry, mean perimeter mean fractal dimension, mean perimeter radius error, mean perimeter texture error, mean perimeter perimeter error, mean perimeter area error, mean perimeter smoothness error, mean perimeter compactness error, mean perimeter concavity error, mean perimeter concave points error, mean perimeter symmetry error, mean perimeter fractal dimension error, mean perimeter worst radius, mean perimeter worst texture, mean perimeter worst perimeter, mean perimeter worst area, mean perimeter worst smoothness, mean perimeter worst compactness, mean perimeter worst concavity, mean perimeter worst concave points, mean perimeter worst symmetry, mean perimeter worst fractal dimension, mean area^2, mean area mean smoothness, mean area mean compactness, mean area mean concavity, mean area mean concave points, mean area mean symmetry, mean area mean fractal dimension, mean area radius error, mean area texture error, mean area perimeter error, mean area area error, mean area smoothness error, mean area compactness error, mean area concavity error, mean area concave points error, mean area symmetry error, mean area fractal dimension error, mean area worst radius, mean area worst texture, mean area worst perimeter, mean area worst area, mean area worst smoothness, mean area worst compactness, mean area worst concavity, mean area worst concave points, mean area worst symmetry, mean area worst fractal dimension, mean smoothness^2, mean smoothness mean compactness, mean smoothness mean concavity, mean smoothness mean concave points, mean smoothness mean symmetry, mean smoothness mean fractal dimension, mean smoothness radius error, mean smoothness texture error, mean smoothness perimeter error, mean smoothness area error, mean smoothness smoothness error, mean smoothness compactness error, mean smoothness concavity error, mean smoothness concave points error, mean smoothness symmetry error, mean smoothness fractal dimension error, mean smoothness worst radius, mean smoothness worst texture, mean smoothness worst perimeter, mean smoothness worst area, mean smoothness worst smoothness, mean smoothness worst compactness, mean smoothness worst concavity, mean smoothness worst concave points, mean smoothness worst symmetry, mean smoothness worst fractal dimension, mean compactness^2, mean compactness mean concavity, mean compactness mean concave points, mean compactness mean symmetry, mean compactness mean fractal dimension, mean compactness radius error, mean compactness texture error, mean compactness perimeter error, mean compactness area error, mean compactness smoothness error, mean compactness compactness error, mean compactness concavity error, mean compactness concave points error, mean compactness symmetry error, mean compactness fractal dimension error, mean compactness worst radius, mean compactness worst texture, mean compactness worst perimeter, mean compactness worst area, mean compactness worst smoothness, mean compactness worst compactness, mean compactness worst concavity, mean compactness worst concave points, mean compactness worst symmetry, mean compactness worst fractal dimension, mean concavity^2, mean concavity mean concave points, mean concavity mean symmetry, mean concavity mean fractal dimension, mean concavity radius error, mean concavity texture error, mean concavity perimeter error, mean concavity area error, mean concavity smoothness error, mean concavity compactness error, mean concavity concavity error, mean concavity concave points error, mean concavity symmetry error, mean concavity fractal dimension error, mean concavity worst radius, mean concavity worst texture, mean concavity worst perimeter, mean concavity worst area, mean concavity worst smoothness, mean concavity worst compactness, mean concavity worst concavity, mean concavity worst concave points, mean concavity worst symmetry, mean concavity worst fractal dimension, mean concave points^2, mean concave points mean symmetry, mean concave points mean fractal dimension, mean concave points radius error, mean concave points texture error, mean concave points perimeter error, mean concave points area error, mean concave points smoothness error, mean concave points compactness error, mean concave points concavity error, mean concave points concave points error, mean concave points symmetry error, mean concave points fractal dimension error, mean concave points worst radius, mean concave points worst texture, mean concave points worst perimeter, mean concave points worst area, mean concave points worst smoothness, mean concave points worst compactness, mean concave points worst concavity, mean concave points worst concave points, mean concave points worst symmetry, mean concave points worst fractal dimension, mean symmetry^2, mean symmetry mean fractal dimension, mean symmetry radius error, mean symmetry texture error, mean symmetry perimeter error, mean symmetry area error, mean symmetry smoothness error, mean symmetry compactness error, mean symmetry concavity error, mean symmetry concave points error, mean symmetry symmetry error, mean symmetry fractal dimension error, mean symmetry worst radius, mean symmetry worst texture, mean symmetry worst perimeter, mean symmetry worst area, mean symmetry worst smoothness, mean symmetry worst compactness, mean symmetry worst concavity, mean symmetry worst concave points, mean symmetry worst symmetry, mean symmetry worst fractal dimension, mean fractal dimension^2, mean fractal dimension radius error, mean fractal dimension texture error, mean fractal dimension perimeter error, mean fractal dimension area error, mean fractal dimension smoothness error, mean fractal dimension compactness error, mean fractal dimension concavity error, mean fractal dimension concave points error, mean fractal dimension symmetry error, mean fractal dimension fractal dimension error, mean fractal dimension worst radius, mean fractal dimension worst texture, mean fractal dimension worst perimeter, mean fractal dimension worst area, mean fractal dimension worst smoothness, mean fractal dimension worst compactness, mean fractal dimension worst concavity, mean fractal dimension worst concave points, mean fractal dimension worst symmetry, mean fractal dimension worst fractal dimension, radius error^2, radius error texture error, radius error perimeter error, radius error area error, radius error smoothness error, radius error compactness error, radius error concavity error, radius error concave points error, radius error symmetry error, radius error fractal dimension error, radius error worst radius, radius error worst texture, radius error worst perimeter, radius error worst area, radius error worst smoothness, radius error worst compactness, radius error worst concavity, radius error worst concave points, radius error worst symmetry, radius error worst fractal dimension, texture error^2, texture error perimeter error, texture error area error, texture error smoothness error, texture error compactness error, texture error concavity error, texture error concave points error, texture error symmetry error, texture error fractal dimension error, texture error worst radius, texture error worst texture, texture error worst perimeter, texture error worst area, texture error worst smoothness, texture error worst compactness, texture error worst concavity, texture error worst concave points, texture error worst symmetry, texture error worst fractal dimension, perimeter error^2, perimeter error area error, perimeter error smoothness error, perimeter error compactness error, perimeter error concavity error, perimeter error concave points error, perimeter error symmetry error, perimeter error fractal dimension error, perimeter error worst radius, perimeter error worst texture, perimeter error worst perimeter, perimeter error worst area, perimeter error worst smoothness, perimeter error worst compactness, perimeter error worst concavity, perimeter error worst concave points, perimeter error worst symmetry, perimeter error worst fractal dimension, area error^2, area error smoothness error, area error compactness error, area error concavity error, area error concave points error, area error symmetry error, area error fractal dimension error, area error worst radius, area error worst texture, area error worst perimeter, area error worst area, area error worst smoothness, area error worst compactness, area error worst concavity, area error worst concave points, area error worst symmetry, area error worst fractal dimension, smoothness error^2, smoothness error compactness error, smoothness error concavity error, smoothness error concave points error, smoothness error symmetry error, smoothness error fractal dimension error, smoothness error worst radius, smoothness error worst texture, smoothness error worst perimeter, smoothness error worst area, smoothness error worst smoothness, smoothness error worst compactness, smoothness error worst concavity, smoothness error worst concave points, smoothness error worst symmetry, smoothness error worst fractal dimension, compactness error^2, compactness error concavity error, compactness error concave points error, compactness error symmetry error, compactness error fractal dimension error, compactness error worst radius, compactness error worst texture, compactness error worst perimeter, compactness error worst area, compactness error worst smoothness, compactness error worst compactness, compactness error worst concavity, compactness error worst concave points, compactness error worst symmetry, compactness error worst fractal dimension, concavity error^2, concavity error concave points error, concavity error symmetry error, concavity error fractal dimension error, concavity error worst radius, concavity error worst texture, concavity error worst perimeter, concavity error worst area, concavity error worst smoothness, concavity error worst compactness, concavity error worst concavity, concavity error worst concave points, concavity error worst symmetry, concavity error worst fractal dimension, concave points error^2, concave points error symmetry error, concave points error fractal dimension error, concave points error worst radius, concave points error worst texture, concave points error worst perimeter, concave points error worst area, concave points error worst smoothness, concave points error worst compactness, concave points error worst concavity, concave points error worst concave points, concave points error worst symmetry, concave points error worst fractal dimension, symmetry error^2, symmetry error fractal dimension error, symmetry error worst radius, symmetry error worst texture, symmetry error worst perimeter, symmetry error worst area, symmetry error worst smoothness, symmetry error worst compactness, symmetry error worst concavity, symmetry error worst concave points, symmetry error worst symmetry, symmetry error worst fractal dimension, fractal dimension error^2, fractal dimension error worst radius, fractal dimension error worst texture, fractal dimension error worst perimeter, fractal dimension error worst area, fractal dimension error worst smoothness, fractal dimension error worst compactness, fractal dimension error worst concavity, fractal dimension error worst concave points, fractal dimension error worst symmetry, fractal dimension error worst fractal dimension, worst radius^2, worst radius worst texture, worst radius worst perimeter, worst radius worst area, worst radius worst smoothness, worst radius worst compactness, worst radius worst concavity, worst radius worst concave points, worst radius worst symmetry, worst radius worst fractal dimension, worst texture^2, worst texture worst perimeter, worst texture worst area, worst texture worst smoothness, worst texture worst compactness, worst texture worst concavity, worst texture worst concave points, worst texture worst symmetry, worst texture worst fractal dimension, worst perimeter^2, worst perimeter worst area, worst perimeter worst smoothness, worst perimeter worst compactness, worst perimeter worst concavity, worst perimeter worst concave points, worst perimeter worst symmetry, worst perimeter worst fractal dimension, worst area^2, worst area worst smoothness, worst area worst compactness, worst area worst concavity, worst area worst concave points, worst area worst symmetry, worst area worst fractal dimension, worst smoothness^2, worst smoothness worst compactness, worst smoothness worst concavity, worst smoothness worst concave points, worst smoothness worst symmetry, worst smoothness worst fractal dimension, worst compactness^2, worst compactness worst concavity, worst compactness worst concave points, worst compactness worst symmetry, worst compactness worst fractal dimension, worst concavity^2, worst concavity worst concave points, worst concavity worst symmetry, worst concavity worst fractal dimension, worst concave points^2, worst concave points worst symmetry, worst concave points worst fractal dimension, worst symmetry^2, worst symmetry worst fractal dimension, worst fractal dimension^2'"
      ]
     },
     "execution_count": 82,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "', '.join(poly.get_feature_names(feature_names))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(455, 495)"
      ]
     },
     "execution_count": 83,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trn_feat.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LinearRegression(copy_X=True, fit_intercept=True, n_jobs=None, normalize=False)"
      ]
     },
     "execution_count": 84,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "regr.fit(trn_feat, y_trn)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(11.37553249805324, 2.5185843777359804)"
      ]
     },
     "execution_count": 85,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "regr_metrics(y_test, regr.predict(poly.fit_transform(test)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "11.7 ms ± 61.1 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)\n"
     ]
    }
   ],
   "source": [
    "%timeit poly.fit_transform(trn)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Experimentos com vetorização e códigos nativos"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {},
   "outputs": [],
   "source": [
    "import math, numpy as np, matplotlib.pyplot as plt\n",
    "from pandas_summary import DataFrameSummary\n",
    "from scipy import ndimage"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {},
   "outputs": [],
   "source": [
    "from numba import jit, vectorize, guvectorize, cuda, float32, void, float64"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "De acordo com [1] todas a utilização dessas bibliotecas vão acelerar o processo de vetorização do método de regressão"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {},
   "outputs": [],
   "source": [
    "def proc_python(xx,yy):\n",
    "    zz = np.zeros(nobs, dtype='float32')\n",
    "    for j in range(nobs):   \n",
    "        x, y = xx[j], yy[j] \n",
    "        x = x*2 - ( y * 55 )\n",
    "        y = x + y*2         \n",
    "        z = x + y + 99      \n",
    "        z = z * ( z - .88 ) \n",
    "        zz[j] = z           \n",
    "    return zz"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {},
   "outputs": [],
   "source": [
    "nobs = 10000\n",
    "x = np.random.randn(nobs).astype('float32')\n",
    "y = np.random.randn(nobs).astype('float32')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "174 ms ± 6.64 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)\n"
     ]
    }
   ],
   "source": [
    "%timeit proc_python(x,y)   # Untyped and unvectorized"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Typed and Vectorized\n",
    "def proc_numpy(x,y):\n",
    "    z = np.zeros(nobs, dtype='float32')\n",
    "    x = x*2 - ( y * 55 )\n",
    "    y = x + y*2         \n",
    "    z = x + y + 99      \n",
    "    z = z * ( z - .88 ) \n",
    "    return z"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 94,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.allclose( proc_numpy(x,y), proc_python(x,y), atol=1e-4 )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "67.8 µs ± 8.95 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)\n"
     ]
    }
   ],
   "source": [
    "%timeit proc_numpy(x,y)    # Typed and vectorized"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Conforme foi demonstrado em [1] podemos comprovar aqui uma diferença considerável em termos de desempenho quando o processo de procura em dados **vetorizados** versus **não vetorizados** \n",
    "\n",
    "Enquanto no primeiro caso tivemos \n",
    "\n",
    "166 ms ± 1.87 por loop\n",
    "\n",
    "No segundo usando \n",
    "\n",
    "~~~python\n",
    "def proc_numpy(x,y): # vectorized\n",
    "~~~\n",
    "\n",
    "Tivemos \n",
    "\n",
    "60.5 µs ± 744 ns por loop"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Agora vamos testar usando **Numba**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {},
   "outputs": [],
   "source": [
    "@jit()\n",
    "def proc_numba(xx,yy,zz):\n",
    "    for j in range(nobs):   \n",
    "        x, y = xx[j], yy[j] \n",
    "        x = x*2 - ( y * 55 )\n",
    "        y = x + y*2         \n",
    "        z = x + y + 99      \n",
    "        z = z * ( z - .88 ) \n",
    "        zz[j] = z           \n",
    "    return zz"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 97,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "z = np.zeros(nobs).astype('float32')\n",
    "np.allclose( proc_numpy(x,y), proc_numba(x,y,z), atol=1e-4 )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "8.38 µs ± 182 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)\n"
     ]
    }
   ],
   "source": [
    "%timeit proc_numba(x,y,z)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Como podemos vizualizar o compilador do Numba otimiza o código de uma maneira mais inteligente do que é possível com Python e Numpy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {},
   "outputs": [],
   "source": [
    "@vectorize\n",
    "def vec_numba(x,y):\n",
    "    x = x*2 - ( y * 55 )\n",
    "    y = x + y*2         \n",
    "    z = x + y + 99      \n",
    "    return z * ( z - .88 ) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 100,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.allclose(vec_numba(x,y), proc_numba(x,y,z), atol=1e-4 )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "8.36 µs ± 20.6 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)\n"
     ]
    }
   ],
   "source": [
    "%timeit vec_numba(x,y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Funções polinomiais com Numba\n",
    "\n",
    "@jit(nopython=True)\n",
    "def vec_poly(x, res):\n",
    "    m,n=x.shape\n",
    "    feat_idx=0\n",
    "    for i in range(n):\n",
    "        v1=x[:,i]\n",
    "        for k in range(m): res[k,feat_idx] = v1[k]\n",
    "        feat_idx+=1\n",
    "        for j in range(i,n):\n",
    "            for k in range(m): res[k,feat_idx] = v1[k]*x[k,j]\n",
    "            feat_idx+=1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {},
   "outputs": [],
   "source": [
    "trn = np.asfortranarray(trn)\n",
    "test = np.asfortranarray(test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {},
   "outputs": [],
   "source": [
    "m,n=trn.shape\n",
    "n_feat = n*(n+1)//2 + n\n",
    "trn_feat = np.zeros((m,n_feat), order='F')\n",
    "test_feat = np.zeros((len(y_test), n_feat), order='F')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {},
   "outputs": [],
   "source": [
    "vec_poly(trn, trn_feat)\n",
    "vec_poly(test, test_feat)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LinearRegression(copy_X=True, fit_intercept=True, n_jobs=None, normalize=False)"
      ]
     },
     "execution_count": 106,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "regr.fit(trn_feat, y_trn)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(11.375541605536771, 2.5185851433818294)"
      ]
     },
     "execution_count": 107,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "regr_metrics(y_test, regr.predict(test_feat))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "206 µs ± 3 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)\n"
     ]
    }
   ],
   "source": [
    "%timeit vec_poly(trn, trn_feat)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "11.9 ms ± 331 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)\n"
     ]
    }
   ],
   "source": [
    "%timeit poly.fit_transform(trn)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Regularização e ruído\n",
    "\n",
    "A regularização é uma maneira de reduzir o excesso de ajuste e criar modelos que generalizem melhor para novos dados."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {},
   "outputs": [],
   "source": [
    "reg_regr = linear_model.LassoCV(n_alphas=10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LassoCV(alphas=None, copy_X=True, cv='warn', eps=0.001, fit_intercept=True,\n",
       "        max_iter=1000, n_alphas=10, n_jobs=None, normalize=False,\n",
       "        positive=False, precompute='auto', random_state=None,\n",
       "        selection='cyclic', tol=0.0001, verbose=False)"
      ]
     },
     "execution_count": 111,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reg_regr.fit(trn_feat, y_trn)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "484.93662302137415"
      ]
     },
     "execution_count": 112,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reg_regr.alpha_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(0.2793076860180617, 0.22432416842595687)"
      ]
     },
     "execution_count": 113,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "regr_metrics(y_test, reg_regr.predict(test_feat))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Ruído\n",
    "\n",
    "Agora vamos adicionar algum ruído aos dados"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {},
   "outputs": [],
   "source": [
    "idxs = np.random.randint(0, len(trn), 10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_trn2 = np.copy(y_trn)\n",
    "y_trn2[idxs] *= 10 # label noise"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(0.2745650835026575, 0.21465805027831458)"
      ]
     },
     "execution_count": 116,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "regr = linear_model.LinearRegression()\n",
    "regr.fit(trn, y_trn)\n",
    "regr_metrics(y_test, regr.predict(test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(0.4050552616858666, 0.3045026155157403)"
      ]
     },
     "execution_count": 117,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "regr.fit(trn, y_trn2)\n",
    "regr_metrics(y_test, regr.predict(test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(0.3452927761520907, 0.25015459075174973)"
      ]
     },
     "execution_count": 119,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hregr = linear_model.HuberRegressor()\n",
    "hregr.fit(trn, y_trn2)\n",
    "regr_metrics(y_test, hregr.predict(test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "De acordo com [1] a perda de huber é uma função de perda menos sensível a valores discrepantes do que a perda de erro ao quadrado. É quadrático para valores de erro pequenos e linear para valores grandes."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}