 Image Classifier.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# In this machine learning project, we will use binary leaf images and extracted features, including shape, margin, and texture to accurately identify plant species using different benchmark classification techniques."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The objective of this machine learning project is to use binary leaf images and extracted features, including shape, margin, and texture, to accurately identify 99 species of plants. Leaves, due to their volume, prevalence, and unique characteristics, are an effective means of differentiating plant species. They also provide a fun introduction to applying techniques that involve image-based features. We are going to apply different classification techniques to benchmark the relevance of classifiers in image classification problem."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#https://s3.amazonaws.com/hackerday.datascience/118/train.csv\n",
    "#https://s3.amazonaws.com/hackerday.datascience/118/test.csv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Image classification\n",
    "\n",
    "# Get all the images\n",
    "\n",
    "# Parse the images and store the pixels in structured data format\n",
    "\n",
    "# remove noise, if present\n",
    "\n",
    "# create models to learn the pattern "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Tea leafs classification\n",
    "\n",
    "# The tea leafs comes through various cartons and with different batches\n",
    "\n",
    "# one individual does a manual inspection\n",
    "\n",
    "# then labeling happens by sampling exercise\n",
    "\n",
    "# High, medium and low quality of leafs that comes to the manufacturer\n",
    "\n",
    "# different tea leafs fetch a differential pricing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Disease classification in plant leafs\n",
    "\n",
    "# each disease in plants have some unique pattern\n",
    "\n",
    "# can we classify them\n",
    "\n",
    "# can we predict them"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# set of images\n",
    "\n",
    "# algorithms\n",
    "\n",
    "# prediction real time or batch mode"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# extraction of pixels and grey scale conversion, that translates the images into a structured data file\n",
    "\n",
    "# by two libraries in Python\n",
    "\n",
    "# scipy library\n",
    "\n",
    "# sklearn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# we do not neeed to parse the images"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# the data set stored in train and test file as below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# training algorithms\n",
    "\n",
    "# Scalable training algorithms if the image is of high pixel size\n",
    "\n",
    "# base algorithms if the image is of low pixel size"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# for large pixel size and high dimensional data, CNN, convolutional neural network\n",
    "\n",
    "# First set of pixxels\n",
    "\n",
    "# apply convolutional layer\n",
    "\n",
    "# represent in another layer\n",
    "\n",
    "# apply another convolution\n",
    "\n",
    "# reduce the representation\n",
    "\n",
    "# apply convolution\n",
    "\n",
    "# manageable size and then train a neural network to solve this\n",
    "\n",
    "# tensorflow, pytorch, azure, GPU processing would be required, if the size of training set is huge"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# if the pixel size is low\n",
    "\n",
    "# use algos from sklearn and scipy to perform classification"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# keras: theano, tensorflow"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "    train.csv - the training set\n",
    "    test.csv - the test set\n",
    "    sample_submission.csv - a sample submission file in the correct format\n",
    "    images - the image files (each image is named with its corresponding id)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "    id - an anonymous id unique to an image\n",
    "    margin_1, margin_2, margin_3, ..., margin_64 - each of the 64 attribute vectors for the margin feature\n",
    "    shape_1, shape_2, shape_3, ..., shape_64 - each of the 64 attribute vectors for the shape feature\n",
    "    texture_1, texture_2, texture_3, ..., texture_64 - each of the 64 attribute vectors for the texture feature\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 273,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 274,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def warn(*args, **kwargs): pass\n",
    "import warnings\n",
    "warnings.warn = warn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 275,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import LabelEncoder\n",
    "from sklearn.cross_validation import StratifiedShuffleSplit"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 276,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "train = pd.read_csv('https://s3.amazonaws.com/hackerday.datascience/118/train.csv')\n",
    "test = pd.read_csv('https://s3.amazonaws.com/hackerday.datascience/118/test.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 277,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Data Preparation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 278,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>species</th>\n",
       "      <th>margin1</th>\n",
       "      <th>margin2</th>\n",
       "      <th>margin3</th>\n",
       "      <th>margin4</th>\n",
       "      <th>margin5</th>\n",
       "      <th>margin6</th>\n",
       "      <th>margin7</th>\n",
       "      <th>margin8</th>\n",
       "      <th>...</th>\n",
       "      <th>texture55</th>\n",
       "      <th>texture56</th>\n",
       "      <th>texture57</th>\n",
       "      <th>texture58</th>\n",
       "      <th>texture59</th>\n",
       "      <th>texture60</th>\n",
       "      <th>texture61</th>\n",
       "      <th>texture62</th>\n",
       "      <th>texture63</th>\n",
       "      <th>texture64</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Acer_Opalus</td>\n",
       "      <td>0.007812</td>\n",
       "      <td>0.023438</td>\n",
       "      <td>0.023438</td>\n",
       "      <td>0.003906</td>\n",
       "      <td>0.011719</td>\n",
       "      <td>0.009766</td>\n",
       "      <td>0.027344</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.007812</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.002930</td>\n",
       "      <td>0.002930</td>\n",
       "      <td>0.035156</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.004883</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.025391</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>Pterocarya_Stenoptera</td>\n",
       "      <td>0.005859</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.031250</td>\n",
       "      <td>0.015625</td>\n",
       "      <td>0.025391</td>\n",
       "      <td>0.001953</td>\n",
       "      <td>0.019531</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000977</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000977</td>\n",
       "      <td>0.023438</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000977</td>\n",
       "      <td>0.039062</td>\n",
       "      <td>0.022461</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>Quercus_Hartwissiana</td>\n",
       "      <td>0.005859</td>\n",
       "      <td>0.009766</td>\n",
       "      <td>0.019531</td>\n",
       "      <td>0.007812</td>\n",
       "      <td>0.003906</td>\n",
       "      <td>0.005859</td>\n",
       "      <td>0.068359</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.154300</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.005859</td>\n",
       "      <td>0.000977</td>\n",
       "      <td>0.007812</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.020508</td>\n",
       "      <td>0.002930</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>5</td>\n",
       "      <td>Tilia_Tomentosa</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.003906</td>\n",
       "      <td>0.023438</td>\n",
       "      <td>0.005859</td>\n",
       "      <td>0.021484</td>\n",
       "      <td>0.019531</td>\n",
       "      <td>0.023438</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000977</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.020508</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.017578</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.047852</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6</td>\n",
       "      <td>Quercus_Variabilis</td>\n",
       "      <td>0.005859</td>\n",
       "      <td>0.003906</td>\n",
       "      <td>0.048828</td>\n",
       "      <td>0.009766</td>\n",
       "      <td>0.013672</td>\n",
       "      <td>0.015625</td>\n",
       "      <td>0.005859</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.096680</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.021484</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.031250</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 194 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   id                species   margin1   margin2   margin3   margin4  \\\n",
       "0   1            Acer_Opalus  0.007812  0.023438  0.023438  0.003906   \n",
       "1   2  Pterocarya_Stenoptera  0.005859  0.000000  0.031250  0.015625   \n",
       "2   3   Quercus_Hartwissiana  0.005859  0.009766  0.019531  0.007812   \n",
       "3   5        Tilia_Tomentosa  0.000000  0.003906  0.023438  0.005859   \n",
       "4   6     Quercus_Variabilis  0.005859  0.003906  0.048828  0.009766   \n",
       "\n",
       "    margin5   margin6   margin7  margin8    ...      texture55  texture56  \\\n",
       "0  0.011719  0.009766  0.027344      0.0    ...       0.007812   0.000000   \n",
       "1  0.025391  0.001953  0.019531      0.0    ...       0.000977   0.000000   \n",
       "2  0.003906  0.005859  0.068359      0.0    ...       0.154300   0.000000   \n",
       "3  0.021484  0.019531  0.023438      0.0    ...       0.000000   0.000977   \n",
       "4  0.013672  0.015625  0.005859      0.0    ...       0.096680   0.000000   \n",
       "\n",
       "   texture57  texture58  texture59  texture60  texture61  texture62  \\\n",
       "0   0.002930   0.002930   0.035156        0.0        0.0   0.004883   \n",
       "1   0.000000   0.000977   0.023438        0.0        0.0   0.000977   \n",
       "2   0.005859   0.000977   0.007812        0.0        0.0   0.000000   \n",
       "3   0.000000   0.000000   0.020508        0.0        0.0   0.017578   \n",
       "4   0.021484   0.000000   0.000000        0.0        0.0   0.000000   \n",
       "\n",
       "   texture63  texture64  \n",
       "0   0.000000   0.025391  \n",
       "1   0.039062   0.022461  \n",
       "2   0.020508   0.002930  \n",
       "3   0.000000   0.047852  \n",
       "4   0.000000   0.031250  \n",
       "\n",
       "[5 rows x 194 columns]"
      ]
     },
     "execution_count": 278,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 279,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Rhododendron_x_Russellianum     10\n",
       "Ilex_Aquifolium                 10\n",
       "Acer_Circinatum                 10\n",
       "Magnolia_Heptapeta              10\n",
       "Quercus_Shumardii               10\n",
       "Quercus_Greggii                 10\n",
       "Ilex_Cornuta                    10\n",
       "Quercus_Palustris               10\n",
       "Alnus_Rubra                     10\n",
       "Lithocarpus_Cleistocarpus       10\n",
       "Lithocarpus_Edulis              10\n",
       "Quercus_Infectoria_sub          10\n",
       "Tilia_Tomentosa                 10\n",
       "Quercus_Alnifolia               10\n",
       "Quercus_Brantii                 10\n",
       "Fagus_Sylvatica                 10\n",
       "Magnolia_Salicifolia            10\n",
       "Olea_Europaea                   10\n",
       "Eucalyptus_Glaucescens          10\n",
       "Alnus_Cordata                   10\n",
       "Acer_Opalus                     10\n",
       "Quercus_Castaneifolia           10\n",
       "Quercus_Dolicholepis            10\n",
       "Tilia_Platyphyllos              10\n",
       "Quercus_Trojana                 10\n",
       "Alnus_Viridis                   10\n",
       "Betula_Pendula                  10\n",
       "Betula_Austrosinensis           10\n",
       "Eucalyptus_Neglecta             10\n",
       "Populus_Nigra                   10\n",
       "                                ..\n",
       "Quercus_Pyrenaica               10\n",
       "Quercus_Suber                   10\n",
       "Celtis_Koraiensis               10\n",
       "Viburnum_x_Rhytidophylloides    10\n",
       "Zelkova_Serrata                 10\n",
       "Liriodendron_Tulipifera         10\n",
       "Acer_Pictum                     10\n",
       "Quercus_Coccinea                10\n",
       "Cornus_Macrophylla              10\n",
       "Quercus_Ellipsoidalis           10\n",
       "Populus_Grandidentata           10\n",
       "Cotinus_Coggygria               10\n",
       "Populus_Adenopoda               10\n",
       "Quercus_x_Turneri               10\n",
       "Quercus_Crassifolia             10\n",
       "Quercus_Phillyraeoides          10\n",
       "Cercis_Siliquastrum             10\n",
       "Viburnum_Tinus                  10\n",
       "Quercus_Cerris                  10\n",
       "Quercus_Agrifolia               10\n",
       "Quercus_Chrysolepis             10\n",
       "Quercus_Rhysophylla             10\n",
       "Quercus_Ilex                    10\n",
       "Acer_Rubrum                     10\n",
       "Quercus_Crassipes               10\n",
       "Quercus_x_Hispanica             10\n",
       "Ulmus_Bergmanniana              10\n",
       "Arundinaria_Simonii             10\n",
       "Alnus_Sieboldiana               10\n",
       "Quercus_Coccifera               10\n",
       "Name: species, Length: 99, dtype: int64"
      ]
     },
     "execution_count": 279,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.species.value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 280,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "le = LabelEncoder().fit(train.species) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 281,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "labels = le.transform(train.species) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 282,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 3, 49, 65, 94, 84, 40, 54, 78, 53, 89, 98, 16, 74, 50, 58, 31, 43,\n",
       "        4, 75, 44, 83, 84, 13, 66, 15,  6, 73, 22, 73, 31, 36, 27, 94, 88,\n",
       "       12, 28, 21, 25, 20, 60, 84, 65, 69, 58, 23, 76, 18, 52, 54,  9, 48,\n",
       "       47, 64, 81, 83, 36, 58, 21, 81, 20, 62, 88, 34, 92, 79, 82, 20, 32,\n",
       "        4, 84, 36, 35, 72, 60, 71, 72, 52, 50, 54, 11, 51, 18, 47,  5,  8,\n",
       "       37, 97, 20, 33,  1, 59,  1, 56,  1,  9, 57, 20, 79, 29, 16, 32, 54,\n",
       "       93, 10, 46, 59, 84, 76, 15, 10, 15,  0, 69,  4, 51, 51, 94, 36, 39,\n",
       "       62,  2, 24, 26, 35, 25, 87,  0, 55, 34, 38,  1, 45,  7, 93, 56, 38,\n",
       "       21, 51, 75, 81, 74, 33, 20, 37,  9, 40, 60, 31, 83, 50, 71, 67, 30,\n",
       "       66,  1, 43, 61, 23, 65, 84, 87, 46, 57, 16,  2, 28, 12, 96, 44, 76,\n",
       "       29, 75, 41, 87, 67, 61, 30,  5, 12, 62,  3, 83, 81,  6, 85,  4, 37,\n",
       "       57, 84, 39, 71, 61,  6, 76, 14, 31, 98, 40, 17, 51, 16, 42, 63, 86,\n",
       "       37, 69, 86, 71, 80, 78, 14, 35, 25,  5, 39,  8,  9, 26, 44, 60, 13,\n",
       "       14, 77, 13, 80, 87, 18, 60, 78, 92, 51, 45, 78, 41, 51, 30, 14, 35,\n",
       "       46, 21,  8,  6, 92, 38, 40, 15, 32, 17, 93, 71, 92, 27, 78, 15, 19,\n",
       "       60, 21, 38, 36, 49, 74, 67, 95, 31, 82, 45, 16, 83, 63, 80, 42, 22,\n",
       "       74, 53, 15, 44, 47, 57, 94, 76, 17, 32, 24, 15, 93, 24, 80, 59, 46,\n",
       "       12, 51, 77, 79, 70, 69, 16,  2, 63, 83, 55, 12, 53,  1, 67,  0,  2,\n",
       "       36, 42, 10,  9, 52, 59,  6, 22, 86, 31, 51, 37, 43, 75, 90, 24, 86,\n",
       "       96, 45, 32, 98, 36, 66, 48, 73, 73, 79, 56, 41, 21, 25, 27, 97, 18,\n",
       "       44, 45, 40, 80, 63, 20, 35,  0,  8, 27, 25, 35, 59, 61, 21, 37, 29,\n",
       "        6, 19, 78, 50, 54, 37, 93, 33, 46, 79, 59, 29, 43,  0, 23, 17, 38,\n",
       "       66, 38, 89, 17, 25, 31, 65, 10, 26, 86, 58, 42, 46, 24, 95, 93,  8,\n",
       "       53, 32, 14, 10, 94,  8,  8, 64, 44, 74, 30, 97, 22, 11, 68, 56, 90,\n",
       "       96, 16, 43, 57, 91, 24, 28, 82, 90, 64, 61, 92, 28, 84, 70, 45, 85,\n",
       "       34,  7, 88, 89, 61, 26, 88, 41, 46,  8, 91, 41, 14, 98, 28, 26, 36,\n",
       "       70, 74,  7, 52, 70, 42, 66, 22, 13, 44, 91, 53, 22, 16, 40, 40, 28,\n",
       "       70,  6, 60, 95, 23, 16, 50, 29, 49,  9, 18, 55, 63, 60, 19, 28, 30,\n",
       "       31, 85, 66, 88, 63, 83, 64, 96, 13, 34, 27, 95, 36, 72, 29, 91, 22,\n",
       "       65, 71, 66, 11, 32,  2, 75, 39,  5, 37, 67, 81, 55, 61, 57, 81, 82,\n",
       "       63, 55, 54, 35, 86, 25, 24, 96, 10, 58, 59, 28, 89, 54, 52, 85, 68,\n",
       "       69,  8, 39, 95, 39, 82, 48, 74, 52, 74, 55,  9, 47, 84, 91, 12, 96,\n",
       "       82, 64,  7, 40, 73, 77, 11, 36, 68, 23, 28, 46, 75, 43,  2, 11, 47,\n",
       "       53, 56, 62, 62, 80, 56, 30,  3, 88, 37, 33, 73, 76, 21,  5, 76, 87,\n",
       "       68, 83, 62, 57, 47, 19, 88, 96, 42, 23, 44, 87, 82, 49, 63, 24, 94,\n",
       "       69, 54,  5, 79, 43, 12, 50,  5, 52, 92,  4, 84,  1, 33, 49, 26, 18,\n",
       "       44, 13, 24, 73, 89, 78, 67, 41, 11, 46, 47, 69,  0, 18, 98, 44, 85,\n",
       "       29, 53,  1, 45,  3,  9, 13,  2, 66, 59, 79,  6, 17, 43, 83, 26,  1,\n",
       "       12, 49, 71, 89, 58, 93, 39, 42, 15, 38, 55, 15, 93,  4, 90, 88, 55,\n",
       "       40, 55, 17, 34, 94, 57, 92, 81, 26, 60, 89, 49, 89, 30, 65, 58,  4,\n",
       "       19,  4, 76, 74, 71, 21, 54, 13, 16, 72, 68, 62, 61, 25, 72,  7, 12,\n",
       "       18, 77, 90, 62, 14,  3, 78, 65, 37, 27, 50, 95, 98, 60, 72, 58, 38,\n",
       "       87, 93, 19,  7, 83, 50,  3, 91, 77,  7, 64, 61, 69, 23, 76, 65, 48,\n",
       "       41, 92, 20, 91, 18, 70,  9,  9, 29, 85, 67,  0, 35, 98, 91, 90, 31,\n",
       "       53, 39, 24, 85, 96, 17,  7, 11, 96, 39, 56, 90, 79, 45, 64, 97, 41,\n",
       "       19, 74, 11, 10, 62, 95, 28, 96, 10,  7, 68,  7, 93, 34, 42, 68, 41,\n",
       "       14, 22, 58, 12, 71, 27, 98, 72, 91,  3, 43, 19, 61, 75, 20, 81, 63,\n",
       "       67, 56, 26, 47, 11, 31, 57, 62, 66, 19, 75, 97, 94, 13, 75, 95, 32,\n",
       "       50, 97, 52, 87, 32,  3, 47, 77, 48, 33, 73, 64, 49, 68, 43, 94, 77,\n",
       "       68, 47, 82,  2, 30, 23, 33, 34, 66, 33, 35, 88, 68, 27, 87, 54, 79,\n",
       "       34, 67, 65, 18,  4, 26, 30, 52, 86,  0, 29, 80, 67, 95, 39, 25, 70,\n",
       "       58, 35, 27, 17, 38, 91, 13, 23, 77, 79, 77, 22, 49, 98, 48, 46, 48,\n",
       "        5, 63, 97, 80, 53, 20, 25, 78, 10, 65, 33, 41, 85, 90, 98, 97, 71,\n",
       "       95, 52,  3, 29, 69, 51, 70, 27, 22, 34,  6, 48, 72, 21, 89, 17, 97,\n",
       "       72, 80, 10, 57, 64, 92, 38, 15, 73, 87, 73, 48, 42, 82, 33, 56,  3,\n",
       "       42,  1, 53, 55, 90, 19,  6, 30, 86, 64, 49,  2,  8, 45, 76, 92,  0,\n",
       "       23, 69, 59, 80, 90, 32,  5, 59, 85, 89, 94, 45, 48, 86, 81, 14,  4,\n",
       "       77, 56, 82,  2, 85, 70, 88,  0, 75, 14, 86, 81, 97, 70, 72, 34, 40,\n",
       "        5, 11, 78, 50])"
      ]
     },
     "execution_count": 282,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 283,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>margin1</th>\n",
       "      <th>margin2</th>\n",
       "      <th>margin3</th>\n",
       "      <th>margin4</th>\n",
       "      <th>margin5</th>\n",
       "      <th>margin6</th>\n",
       "      <th>margin7</th>\n",
       "      <th>margin8</th>\n",
       "      <th>margin9</th>\n",
       "      <th>...</th>\n",
       "      <th>texture55</th>\n",
       "      <th>texture56</th>\n",
       "      <th>texture57</th>\n",
       "      <th>texture58</th>\n",
       "      <th>texture59</th>\n",
       "      <th>texture60</th>\n",
       "      <th>texture61</th>\n",
       "      <th>texture62</th>\n",
       "      <th>texture63</th>\n",
       "      <th>texture64</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>4</td>\n",
       "      <td>0.019531</td>\n",
       "      <td>0.009766</td>\n",
       "      <td>0.078125</td>\n",
       "      <td>0.011719</td>\n",
       "      <td>0.003906</td>\n",
       "      <td>0.015625</td>\n",
       "      <td>0.005859</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.005859</td>\n",
       "      <td>...</td>\n",
       "      <td>0.006836</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.015625</td>\n",
       "      <td>0.000977</td>\n",
       "      <td>0.015625</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.003906</td>\n",
       "      <td>0.053711</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>7</td>\n",
       "      <td>0.007812</td>\n",
       "      <td>0.005859</td>\n",
       "      <td>0.064453</td>\n",
       "      <td>0.009766</td>\n",
       "      <td>0.003906</td>\n",
       "      <td>0.013672</td>\n",
       "      <td>0.007812</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.033203</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.006836</td>\n",
       "      <td>0.001953</td>\n",
       "      <td>0.013672</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000977</td>\n",
       "      <td>0.037109</td>\n",
       "      <td>0.044922</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>9</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.001953</td>\n",
       "      <td>0.021484</td>\n",
       "      <td>0.041016</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.023438</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.011719</td>\n",
       "      <td>...</td>\n",
       "      <td>0.128910</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000977</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.015625</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>12</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.009766</td>\n",
       "      <td>0.011719</td>\n",
       "      <td>0.017578</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.003906</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.003906</td>\n",
       "      <td>...</td>\n",
       "      <td>0.012695</td>\n",
       "      <td>0.015625</td>\n",
       "      <td>0.002930</td>\n",
       "      <td>0.036133</td>\n",
       "      <td>0.013672</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.089844</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.008789</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>13</td>\n",
       "      <td>0.001953</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.015625</td>\n",
       "      <td>0.009766</td>\n",
       "      <td>0.039062</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.009766</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.005859</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.042969</td>\n",
       "      <td>0.016602</td>\n",
       "      <td>0.010742</td>\n",
       "      <td>0.041016</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.007812</td>\n",
       "      <td>0.009766</td>\n",
       "      <td>0.007812</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 193 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   id   margin1   margin2   margin3   margin4   margin5   margin6   margin7  \\\n",
       "0   4  0.019531  0.009766  0.078125  0.011719  0.003906  0.015625  0.005859   \n",
       "1   7  0.007812  0.005859  0.064453  0.009766  0.003906  0.013672  0.007812   \n",
       "2   9  0.000000  0.000000  0.001953  0.021484  0.041016  0.000000  0.023438   \n",
       "3  12  0.000000  0.000000  0.009766  0.011719  0.017578  0.000000  0.003906   \n",
       "4  13  0.001953  0.000000  0.015625  0.009766  0.039062  0.000000  0.009766   \n",
       "\n",
       "   margin8   margin9    ...      texture55  texture56  texture57  texture58  \\\n",
       "0      0.0  0.005859    ...       0.006836   0.000000   0.015625   0.000977   \n",
       "1      0.0  0.033203    ...       0.000000   0.000000   0.006836   0.001953   \n",
       "2      0.0  0.011719    ...       0.128910   0.000000   0.000977   0.000000   \n",
       "3      0.0  0.003906    ...       0.012695   0.015625   0.002930   0.036133   \n",
       "4      0.0  0.005859    ...       0.000000   0.042969   0.016602   0.010742   \n",
       "\n",
       "   texture59  texture60  texture61  texture62  texture63  texture64  \n",
       "0   0.015625        0.0        0.0   0.000000   0.003906   0.053711  \n",
       "1   0.013672        0.0        0.0   0.000977   0.037109   0.044922  \n",
       "2   0.000000        0.0        0.0   0.015625   0.000000   0.000000  \n",
       "3   0.013672        0.0        0.0   0.089844   0.000000   0.008789  \n",
       "4   0.041016        0.0        0.0   0.007812   0.009766   0.007812  \n",
       "\n",
       "[5 rows x 193 columns]"
      ]
     },
     "execution_count": 283,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 284,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 990 entries, 0 to 989\n",
      "Columns: 194 entries, id to texture64\n",
      "dtypes: float64(192), int64(1), object(1)\n",
      "memory usage: 1.5+ MB\n"
     ]
    }
   ],
   "source": [
    "train.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 285,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 594 entries, 0 to 593\n",
      "Columns: 193 entries, id to texture64\n",
      "dtypes: float64(192), int64(1)\n",
      "memory usage: 895.7 KB\n"
     ]
    }
   ],
   "source": [
    "test.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 286,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# a function to organize both training and test dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 287,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def encode(train, test):\n",
    "    le = LabelEncoder().fit(train.species) \n",
    "    labels = le.transform(train.species)           # encode species strings\n",
    "    classes = list(le.classes_)                    \n",
    "    test_ids = test.id                             \n",
    "    \n",
    "    train = train.drop(['species', 'id'], axis=1)  \n",
    "    test = test.drop(['id'], axis=1)\n",
    "    \n",
    "    return train, labels, test, test_ids, classes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 288,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>margin1</th>\n",
       "      <th>margin2</th>\n",
       "      <th>margin3</th>\n",
       "      <th>margin4</th>\n",
       "      <th>margin5</th>\n",
       "      <th>margin6</th>\n",
       "      <th>margin7</th>\n",
       "      <th>margin8</th>\n",
       "      <th>margin9</th>\n",
       "      <th>margin10</th>\n",
       "      <th>...</th>\n",
       "      <th>texture55</th>\n",
       "      <th>texture56</th>\n",
       "      <th>texture57</th>\n",
       "      <th>texture58</th>\n",
       "      <th>texture59</th>\n",
       "      <th>texture60</th>\n",
       "      <th>texture61</th>\n",
       "      <th>texture62</th>\n",
       "      <th>texture63</th>\n",
       "      <th>texture64</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.007812</td>\n",
       "      <td>0.023438</td>\n",
       "      <td>0.023438</td>\n",
       "      <td>0.003906</td>\n",
       "      <td>0.011719</td>\n",
       "      <td>0.009766</td>\n",
       "      <td>0.027344</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.001953</td>\n",
       "      <td>0.033203</td>\n",
       "      <td>...</td>\n",
       "      <td>0.007812</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.002930</td>\n",
       "      <td>0.002930</td>\n",
       "      <td>0.035156</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.004883</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.025391</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.005859</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.031250</td>\n",
       "      <td>0.015625</td>\n",
       "      <td>0.025391</td>\n",
       "      <td>0.001953</td>\n",
       "      <td>0.019531</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.007812</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000977</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000977</td>\n",
       "      <td>0.023438</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000977</td>\n",
       "      <td>0.039062</td>\n",
       "      <td>0.022461</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.005859</td>\n",
       "      <td>0.009766</td>\n",
       "      <td>0.019531</td>\n",
       "      <td>0.007812</td>\n",
       "      <td>0.003906</td>\n",
       "      <td>0.005859</td>\n",
       "      <td>0.068359</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.044922</td>\n",
       "      <td>...</td>\n",
       "      <td>0.154300</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.005859</td>\n",
       "      <td>0.000977</td>\n",
       "      <td>0.007812</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.020508</td>\n",
       "      <td>0.002930</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.003906</td>\n",
       "      <td>0.023438</td>\n",
       "      <td>0.005859</td>\n",
       "      <td>0.021484</td>\n",
       "      <td>0.019531</td>\n",
       "      <td>0.023438</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.013672</td>\n",
       "      <td>0.017578</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000977</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.020508</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.017578</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.047852</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.005859</td>\n",
       "      <td>0.003906</td>\n",
       "      <td>0.048828</td>\n",
       "      <td>0.009766</td>\n",
       "      <td>0.013672</td>\n",
       "      <td>0.015625</td>\n",
       "      <td>0.005859</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.005859</td>\n",
       "      <td>...</td>\n",
       "      <td>0.096680</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.021484</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.031250</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 192 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "    margin1   margin2   margin3   margin4   margin5   margin6   margin7  \\\n",
       "0  0.007812  0.023438  0.023438  0.003906  0.011719  0.009766  0.027344   \n",
       "1  0.005859  0.000000  0.031250  0.015625  0.025391  0.001953  0.019531   \n",
       "2  0.005859  0.009766  0.019531  0.007812  0.003906  0.005859  0.068359   \n",
       "3  0.000000  0.003906  0.023438  0.005859  0.021484  0.019531  0.023438   \n",
       "4  0.005859  0.003906  0.048828  0.009766  0.013672  0.015625  0.005859   \n",
       "\n",
       "   margin8   margin9  margin10    ...      texture55  texture56  texture57  \\\n",
       "0      0.0  0.001953  0.033203    ...       0.007812   0.000000   0.002930   \n",
       "1      0.0  0.000000  0.007812    ...       0.000977   0.000000   0.000000   \n",
       "2      0.0  0.000000  0.044922    ...       0.154300   0.000000   0.005859   \n",
       "3      0.0  0.013672  0.017578    ...       0.000000   0.000977   0.000000   \n",
       "4      0.0  0.000000  0.005859    ...       0.096680   0.000000   0.021484   \n",
       "\n",
       "   texture58  texture59  texture60  texture61  texture62  texture63  texture64  \n",
       "0   0.002930   0.035156        0.0        0.0   0.004883   0.000000   0.025391  \n",
       "1   0.000977   0.023438        0.0        0.0   0.000977   0.039062   0.022461  \n",
       "2   0.000977   0.007812        0.0        0.0   0.000000   0.020508   0.002930  \n",
       "3   0.000000   0.020508        0.0        0.0   0.017578   0.000000   0.047852  \n",
       "4   0.000000   0.000000        0.0        0.0   0.000000   0.000000   0.031250  \n",
       "\n",
       "[5 rows x 192 columns]"
      ]
     },
     "execution_count": 288,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train, labels, test, test_ids, classes = encode(train, test)\n",
    "train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 289,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# stratified sampling rather than simple random sampling"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 290,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#Stratification is necessary for this dataset because there is a relatively large number of classes \n",
    "#(99 classes for 990 samples). This will ensure we have all classes represented in both the train and test indices."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 291,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "sss = StratifiedShuffleSplit(labels, 10, test_size=0.2, random_state=1243)\n",
    "\n",
    "for train_index, test_index in sss:\n",
    "    X_train, X_test = train.values[train_index], train.values[test_index]\n",
    "    y_train, y_test = labels[train_index], labels[test_index]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 292,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(792, 192) (792,)\n",
      "(198, 192) (198,)\n"
     ]
    }
   ],
   "source": [
    "print(X_train.shape,y_train.shape)\n",
    "print(X_test.shape, y_test.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 293,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#selection of classifiers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 294,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.metrics import accuracy_score, log_loss\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from sklearn.svm import SVC, LinearSVC, NuSVC\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "from sklearn.ensemble import RandomForestClassifier, AdaBoostClassifier, GradientBoostingClassifier\n",
    "from sklearn.naive_bayes import GaussianNB\n",
    "from sklearn.discriminant_analysis import LinearDiscriminantAnalysis\n",
    "from sklearn.discriminant_analysis import QuadraticDiscriminantAnalysis\n",
    "from sklearn.metrics import confusion_matrix, classification_report"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 295,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# initialize all the classifiers\n",
    "classifiers = [\n",
    "    KNeighborsClassifier(3),\n",
    "    SVC(kernel=\"rbf\", C=0.025, probability=True),\n",
    "    NuSVC(probability=True),\n",
    "    DecisionTreeClassifier(),\n",
    "    RandomForestClassifier(bootstrap=True, class_weight=None, criterion='gini',\n",
    "             max_depth=None, max_features='auto', max_leaf_nodes=None,\n",
    "             min_impurity_decrease=0.0, min_impurity_split=None,\n",
    "             min_samples_leaf=1, min_samples_split=2,\n",
    "             min_weight_fraction_leaf=0.0, n_estimators=10, n_jobs=1,\n",
    "             oob_score=False, random_state=None, verbose=0,\n",
    "             warm_start=False),\n",
    "    AdaBoostClassifier(),\n",
    "    GradientBoostingClassifier(),\n",
    "    GaussianNB(),\n",
    "    LinearDiscriminantAnalysis(),\n",
    "    QuadraticDiscriminantAnalysis()]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# initialize all the classifiers with the best parameters from grid search\n",
    "classifiers = [\n",
    "    KNeighborsClassifier(algorithm='auto', leaf_size=30, metric='minkowski',\n",
    "            metric_params=None, n_jobs=1, n_neighbors=3, p=2,\n",
    "            weights='uniform'),\n",
    "    SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0,\n",
    "  decision_function_shape='ovr', degree=3, gamma='auto', kernel='linear',\n",
    "  max_iter=-1, probability=False, random_state=None, shrinking=True,\n",
    "  tol=0.001, verbose=False),\n",
    "    NuSVC(cache_size=200, class_weight=None, coef0=0.0,\n",
    "   decision_function_shape='ovr', degree=3, gamma='auto', kernel='linear',\n",
    "   max_iter=-1, nu=0.05, probability=False, random_state=None,\n",
    "   shrinking=True, tol=0.001, verbose=False),\n",
    "    DecisionTreeClassifier(),\n",
    "    RandomForestClassifier(bootstrap=True, class_weight=None, criterion='gini',\n",
    "             max_depth=None, max_features='auto', max_leaf_nodes=None,\n",
    "             min_impurity_decrease=0.0, min_impurity_split=None,\n",
    "             min_samples_leaf=1, min_samples_split=2,\n",
    "             min_weight_fraction_leaf=0.0, n_estimators=10, n_jobs=1,\n",
    "             oob_score=False, random_state=None, verbose=0,\n",
    "             warm_start=False),\n",
    "    AdaBoostClassifier(),\n",
    "    GradientBoostingClassifier(),\n",
    "    GaussianNB(),\n",
    "    LinearDiscriminantAnalysis(),\n",
    "    QuadraticDiscriminantAnalysis()]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 296,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[KNeighborsClassifier(algorithm='auto', leaf_size=30, metric='minkowski',\n",
       "            metric_params=None, n_jobs=1, n_neighbors=3, p=2,\n",
       "            weights='uniform'),\n",
       " SVC(C=0.025, cache_size=200, class_weight=None, coef0=0.0,\n",
       "   decision_function_shape='ovr', degree=3, gamma='auto', kernel='rbf',\n",
       "   max_iter=-1, probability=True, random_state=None, shrinking=True,\n",
       "   tol=0.001, verbose=False),\n",
       " NuSVC(cache_size=200, class_weight=None, coef0=0.0,\n",
       "    decision_function_shape='ovr', degree=3, gamma='auto', kernel='rbf',\n",
       "    max_iter=-1, nu=0.5, probability=True, random_state=None,\n",
       "    shrinking=True, tol=0.001, verbose=False),\n",
       " DecisionTreeClassifier(class_weight=None, criterion='gini', max_depth=None,\n",
       "             max_features=None, max_leaf_nodes=None,\n",
       "             min_impurity_decrease=0.0, min_impurity_split=None,\n",
       "             min_samples_leaf=1, min_samples_split=2,\n",
       "             min_weight_fraction_leaf=0.0, presort=False, random_state=None,\n",
       "             splitter='best'),\n",
       " RandomForestClassifier(bootstrap=True, class_weight=None, criterion='gini',\n",
       "             max_depth=None, max_features='auto', max_leaf_nodes=None,\n",
       "             min_impurity_decrease=0.0, min_impurity_split=None,\n",
       "             min_samples_leaf=1, min_samples_split=2,\n",
       "             min_weight_fraction_leaf=0.0, n_estimators=10, n_jobs=1,\n",
       "             oob_score=False, random_state=None, verbose=0,\n",
       "             warm_start=False),\n",
       " AdaBoostClassifier(algorithm='SAMME.R', base_estimator=None,\n",
       "           learning_rate=1.0, n_estimators=50, random_state=None),\n",
       " GradientBoostingClassifier(criterion='friedman_mse', init=None,\n",
       "               learning_rate=0.1, loss='deviance', max_depth=3,\n",
       "               max_features=None, max_leaf_nodes=None,\n",
       "               min_impurity_decrease=0.0, min_impurity_split=None,\n",
       "               min_samples_leaf=1, min_samples_split=2,\n",
       "               min_weight_fraction_leaf=0.0, n_estimators=100,\n",
       "               presort='auto', random_state=None, subsample=1.0, verbose=0,\n",
       "               warm_start=False),\n",
       " GaussianNB(priors=None),\n",
       " LinearDiscriminantAnalysis(n_components=None, priors=None, shrinkage=None,\n",
       "               solver='svd', store_covariance=False, tol=0.0001),\n",
       " QuadraticDiscriminantAnalysis(priors=None, reg_param=0.0,\n",
       "                store_covariance=False, store_covariances=None, tol=0.0001)]"
      ]
     },
     "execution_count": 296,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "classifiers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 297,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Logging for Visual Comparison\n",
    "log_cols=[\"Classifier\", \"Accuracy\", \"Log Loss\"]\n",
    "log = pd.DataFrame(columns=log_cols)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 298,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "==============================\n",
      "KNeighborsClassifier\n",
      "****Results****\n",
      "Accuracy: 91.4141%\n",
      "Log Loss: 1.5688376780105346\n",
      "==============================\n",
      "SVC\n",
      "****Results****\n",
      "Accuracy: 80.8081%\n",
      "Log Loss: 4.606697723541745\n",
      "==============================\n",
      "NuSVC\n",
      "****Results****\n",
      "Accuracy: 88.8889%\n",
      "Log Loss: 2.488363494528804\n",
      "==============================\n",
      "DecisionTreeClassifier\n",
      "****Results****\n",
      "Accuracy: 67.1717%\n",
      "Log Loss: 11.338487200349565\n",
      "==============================\n",
      "RandomForestClassifier\n",
      "****Results****\n",
      "Accuracy: 88.8889%\n",
      "Log Loss: 1.2773702484656477\n",
      "==============================\n",
      "AdaBoostClassifier\n",
      "****Results****\n",
      "Accuracy: 3.5354%\n",
      "Log Loss: 4.65598671006233\n",
      "==============================\n",
      "GradientBoostingClassifier\n",
      "****Results****\n",
      "Accuracy: 58.0808%\n",
      "Log Loss: 2.8614569297893144\n",
      "==============================\n",
      "GaussianNB\n",
      "****Results****\n",
      "Accuracy: 57.0707%\n",
      "Log Loss: 14.827252492764792\n",
      "==============================\n",
      "LinearDiscriminantAnalysis\n",
      "****Results****\n",
      "Accuracy: 98.4848%\n",
      "Log Loss: 0.9877283476605773\n",
      "==============================\n",
      "QuadraticDiscriminantAnalysis\n",
      "****Results****\n",
      "Accuracy: 1.5152%\n",
      "Log Loss: 34.01546160104849\n",
      "==============================\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Applications/anaconda3/envs/Python36/lib/python3.6/site-packages/sklearn/discriminant_analysis.py:510: RuntimeWarning: overflow encountered in exp\n",
      "  np.exp(prob, prob)\n"
     ]
    }
   ],
   "source": [
    "for clf in classifiers:\n",
    "    clf.fit(X_train, y_train)\n",
    "    name = clf.__class__.__name__\n",
    "    \n",
    "    print(\"=\"*30)\n",
    "    print(name)\n",
    "    \n",
    "    print('****Results****')\n",
    "    train_predictions = clf.predict(X_test)\n",
    "    acc = accuracy_score(y_test, train_predictions)\n",
    "    print(\"Accuracy: {:.4%}\".format(acc))\n",
    "    \n",
    "    train_predictions = clf.predict_proba(X_test)\n",
    "    ll = log_loss(y_test, train_predictions)\n",
    "    print(\"Log Loss: {}\".format(ll))\n",
    "    \n",
    "    log_entry = pd.DataFrame([[name, acc*100, ll]], columns=log_cols)\n",
    "    log = log.append(log_entry)\n",
    "    \n",
    "print(\"=\"*30)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 173,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# initialize all the classifiers\n",
    "classifiers = [\n",
    "    KNeighborsClassifier(3),\n",
    "    SVC(kernel=\"rbf\", C=0.025, probability=True),\n",
    "    NuSVC(probability=True),\n",
    "    DecisionTreeClassifier(),\n",
    "    RandomForestClassifier(),\n",
    "    AdaBoostClassifier(),\n",
    "    GradientBoostingClassifier(),\n",
    "    GaussianNB(),\n",
    "    LinearDiscriminantAnalysis(),\n",
    "    QuadraticDiscriminantAnalysis()]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 311,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.cross_validation import ShuffleSplit\n",
    "from sklearn.grid_search import GridSearchCV\n",
    "def fit_model(X,y):\n",
    "    cv_sets = ShuffleSplit(X_train.shape[0],n_iter=10,test_size=0.20,random_state=1234)\n",
    "    sv = SVC()\n",
    "    params = {'kernel':('linear','poly','sigmoid','rbf'),\n",
    "              'C':[0.01,0.05,0.025,0.07,0.09,1.0]\n",
    "              }\n",
    "    grid = GridSearchCV(sv,params,cv=cv_sets)\n",
    "    grid = grid.fit(X,y)\n",
    "    return grid.best_estimator_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 312,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0,\n",
       "  decision_function_shape='ovr', degree=3, gamma='auto', kernel='linear',\n",
       "  max_iter=-1, probability=False, random_state=None, shrinking=True,\n",
       "  tol=0.001, verbose=False)"
      ]
     },
     "execution_count": 312,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fit_model(X_train,y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 307,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.cross_validation import ShuffleSplit\n",
    "from sklearn.grid_search import GridSearchCV\n",
    "def fit_model(X,y):\n",
    "    cv_sets = ShuffleSplit(X_train.shape[0],n_iter=10,test_size=0.20,random_state=1234)\n",
    "    nsv = NuSVC()\n",
    "    params = {'kernel':('linear','poly','sigmoid','rbf'),\n",
    "              'nu':[0.01,0.05,0.025]\n",
    "              }\n",
    "    grid = GridSearchCV(nsv,params,cv=cv_sets)\n",
    "    grid = grid.fit(X,y)\n",
    "    return grid.best_estimator_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 308,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "NuSVC(cache_size=200, class_weight=None, coef0=0.0,\n",
       "   decision_function_shape='ovr', degree=3, gamma='auto', kernel='linear',\n",
       "   max_iter=-1, nu=0.05, probability=False, random_state=None,\n",
       "   shrinking=True, tol=0.001, verbose=False)"
      ]
     },
     "execution_count": 308,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fit_model(X_train,y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 313,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.cross_validation import ShuffleSplit\n",
    "from sklearn.grid_search import GridSearchCV\n",
    "def fit_model(X,y):\n",
    "    cv_sets = ShuffleSplit(X_train.shape[0],n_iter=10,test_size=0.20,random_state=1234)\n",
    "    dt = DecisionTreeClassifier()\n",
    "    params = {'criterion':('gini','entropy'),\n",
    "              'max_depth':[2,3,4,5]\n",
    "              }\n",
    "    grid = GridSearchCV(dt,params,cv=cv_sets)\n",
    "    grid = grid.fit(X,y)\n",
    "    return grid.best_estimator_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 314,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DecisionTreeClassifier(class_weight=None, criterion='entropy', max_depth=5,\n",
       "            max_features=None, max_leaf_nodes=None,\n",
       "            min_impurity_decrease=0.0, min_impurity_split=None,\n",
       "            min_samples_leaf=1, min_samples_split=2,\n",
       "            min_weight_fraction_leaf=0.0, presort=False, random_state=None,\n",
       "            splitter='best')"
      ]
     },
     "execution_count": 314,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fit_model(X_train,y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 315,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.cross_validation import ShuffleSplit\n",
    "from sklearn.grid_search import GridSearchCV\n",
    "def fit_model(X,y):\n",
    "    cv_sets = ShuffleSplit(X_train.shape[0],n_iter=10,test_size=0.20,random_state=1234)\n",
    "    rf = RandomForestClassifier()\n",
    "    params = {'criterion':('gini','entropy'),\n",
    "              'n_estimators':[100,200,300,500]\n",
    "              }\n",
    "    grid = GridSearchCV(rf,params,cv=cv_sets)\n",
    "    grid = grid.fit(X,y)\n",
    "    return grid.best_estimator_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 316,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RandomForestClassifier(bootstrap=True, class_weight=None, criterion='gini',\n",
       "            max_depth=None, max_features='auto', max_leaf_nodes=None,\n",
       "            min_impurity_decrease=0.0, min_impurity_split=None,\n",
       "            min_samples_leaf=1, min_samples_split=2,\n",
       "            min_weight_fraction_leaf=0.0, n_estimators=500, n_jobs=1,\n",
       "            oob_score=False, random_state=None, verbose=0,\n",
       "            warm_start=False)"
      ]
     },
     "execution_count": 316,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fit_model(X_train,y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 319,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.cross_validation import ShuffleSplit\n",
    "from sklearn.grid_search import GridSearchCV\n",
    "def fit_model(X,y):\n",
    "    cv_sets = ShuffleSplit(X_train.shape[0],n_iter=10,test_size=0.20,random_state=1234)\n",
    "    ab = AdaBoostClassifier()\n",
    "    params = {'algorithm':('SAMME', 'SAMME.R'),\n",
    "              'n_estimators':[100,200]\n",
    "              }\n",
    "    grid = GridSearchCV(ab,params,cv=cv_sets)\n",
    "    grid = grid.fit(X,y)\n",
    "    return grid.best_estimator_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 320,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "AdaBoostClassifier(algorithm='SAMME', base_estimator=None, learning_rate=1.0,\n",
       "          n_estimators=200, random_state=None)"
      ]
     },
     "execution_count": 320,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fit_model(X_train,y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 327,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.cross_validation import ShuffleSplit\n",
    "from sklearn.grid_search import GridSearchCV\n",
    "def fit_model(X,y):\n",
    "    cv_sets = ShuffleSplit(X_train.shape[0],n_iter=10,test_size=0.20,random_state=1234)\n",
    "    gb = GradientBoostingClassifier()\n",
    "    params = {\n",
    "              'n_estimators':[100,200]\n",
    "              }\n",
    "    grid = GridSearchCV(gb,params,cv=cv_sets)\n",
    "    grid = grid.fit(X,y)\n",
    "    return grid.best_estimator_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 328,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GradientBoostingClassifier(criterion='friedman_mse', init=None,\n",
       "              learning_rate=0.1, loss='deviance', max_depth=3,\n",
       "              max_features=None, max_leaf_nodes=None,\n",
       "              min_impurity_decrease=0.0, min_impurity_split=None,\n",
       "              min_samples_leaf=1, min_samples_split=2,\n",
       "              min_weight_fraction_leaf=0.0, n_estimators=100,\n",
       "              presort='auto', random_state=None, subsample=1.0, verbose=0,\n",
       "              warm_start=False)"
      ]
     },
     "execution_count": 328,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fit_model(X_train,y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 338,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.cross_validation import ShuffleSplit\n",
    "from sklearn.grid_search import GridSearchCV\n",
    "def fit_model(X,y):\n",
    "    cv_sets = ShuffleSplit(X_train.shape[0],n_iter=10,test_size=0.20,random_state=1234)\n",
    "    lda = LinearDiscriminantAnalysis()\n",
    "    params = {'solver' :('svd','eigen','lsqr')\n",
    "              }\n",
    "    grid = GridSearchCV(lda,params,cv=cv_sets)\n",
    "    grid = grid.fit(X,y)\n",
    "    return grid.best_estimator_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 339,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LinearDiscriminantAnalysis(n_components=None, priors=None, shrinkage=None,\n",
       "              solver='svd', store_covariance=False, tol=0.0001)"
      ]
     },
     "execution_count": 339,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fit_model(X_train,y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 340,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# initialize all the classifiers with the best parameters from grid search\n",
    "classifiers = [\n",
    "    KNeighborsClassifier(algorithm='auto', leaf_size=30, metric='minkowski',\n",
    "            metric_params=None, n_jobs=1, n_neighbors=3, p=2,\n",
    "            weights='uniform'),\n",
    "    \n",
    "    SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0,\n",
    "  decision_function_shape='ovr', degree=3, gamma='auto', kernel='linear',\n",
    "  max_iter=-1, probability=True, random_state=None, shrinking=True,\n",
    "  tol=0.001, verbose=False),\n",
    "    \n",
    "    NuSVC(cache_size=200, class_weight=None, coef0=0.0,\n",
    "   decision_function_shape='ovr', degree=3, gamma='auto', kernel='linear',\n",
    "   max_iter=-1, nu=0.05, probability=True, random_state=None,\n",
    "   shrinking=True, tol=0.001, verbose=False),\n",
    "    \n",
    "    DecisionTreeClassifier(class_weight=None, criterion='entropy', max_depth=5,\n",
    "            max_features=None, max_leaf_nodes=None,\n",
    "            min_impurity_decrease=0.0, min_impurity_split=None,\n",
    "            min_samples_leaf=1, min_samples_split=2,\n",
    "            min_weight_fraction_leaf=0.0, presort=False, random_state=None,\n",
    "            splitter='best'),\n",
    "    \n",
    "    RandomForestClassifier(bootstrap=True, class_weight=None, criterion='gini',\n",
    "            max_depth=None, max_features='auto', max_leaf_nodes=None,\n",
    "            min_impurity_decrease=0.0, min_impurity_split=None,\n",
    "            min_samples_leaf=1, min_samples_split=2,\n",
    "            min_weight_fraction_leaf=0.0, n_estimators=500, n_jobs=1,\n",
    "            oob_score=False, random_state=None, verbose=0,\n",
    "            warm_start=False),\n",
    "    \n",
    "    AdaBoostClassifier(algorithm='SAMME', base_estimator=None, learning_rate=1.0,\n",
    "          n_estimators=200, random_state=None),\n",
    "    \n",
    "    GradientBoostingClassifier(criterion='friedman_mse', init=None,\n",
    "              learning_rate=0.1, loss='deviance', max_depth=3,\n",
    "              max_features=None, max_leaf_nodes=None,\n",
    "              min_impurity_decrease=0.0, min_impurity_split=None,\n",
    "              min_samples_leaf=1, min_samples_split=2,\n",
    "              min_weight_fraction_leaf=0.0, n_estimators=100,\n",
    "              presort='auto', random_state=None, subsample=1.0, verbose=0,\n",
    "              warm_start=False),\n",
    "    \n",
    "    GaussianNB(),\n",
    "    \n",
    "    LinearDiscriminantAnalysis(n_components=None, priors=None, shrinkage=None,\n",
    "              solver='svd', store_covariance=False, tol=0.0001),\n",
    "    \n",
    "    QuadraticDiscriminantAnalysis()]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 341,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Logging for Visual Comparison\n",
    "log_cols=[\"Classifier\", \"Accuracy\", \"Log Loss\"]\n",
    "log = pd.DataFrame(columns=log_cols)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 342,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "==============================\n",
      "KNeighborsClassifier\n",
      "****Results****\n",
      "Accuracy: 91.4141%\n",
      "Log Loss: 1.5688376780105346\n",
      "==============================\n",
      "SVC\n",
      "****Results****\n",
      "Accuracy: 80.8081%\n",
      "Log Loss: 4.309221427609576\n",
      "==============================\n",
      "NuSVC\n",
      "****Results****\n",
      "Accuracy: 97.9798%\n",
      "Log Loss: 2.274298969635117\n",
      "==============================\n",
      "DecisionTreeClassifier\n",
      "****Results****\n",
      "Accuracy: 25.7576%\n",
      "Log Loss: 6.08538919298582\n",
      "==============================\n",
      "RandomForestClassifier\n",
      "****Results****\n",
      "Accuracy: 98.9899%\n",
      "Log Loss: 0.7379722298231588\n",
      "==============================\n",
      "AdaBoostClassifier\n",
      "****Results****\n",
      "Accuracy: 7.5758%\n",
      "Log Loss: 4.595045293889405\n",
      "==============================\n",
      "GradientBoostingClassifier\n",
      "****Results****\n",
      "Accuracy: 58.0808%\n",
      "Log Loss: 2.8015768618879817\n",
      "==============================\n",
      "GaussianNB\n",
      "****Results****\n",
      "Accuracy: 57.0707%\n",
      "Log Loss: 14.827252492764792\n",
      "==============================\n",
      "LinearDiscriminantAnalysis\n",
      "****Results****\n",
      "Accuracy: 98.4848%\n",
      "Log Loss: 0.9877283476605773\n",
      "==============================\n",
      "QuadraticDiscriminantAnalysis\n",
      "****Results****\n",
      "Accuracy: 1.5152%\n",
      "Log Loss: 34.01546160104849\n",
      "==============================\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Applications/anaconda3/envs/Python36/lib/python3.6/site-packages/sklearn/discriminant_analysis.py:510: RuntimeWarning: overflow encountered in exp\n",
      "  np.exp(prob, prob)\n"
     ]
    }
   ],
   "source": [
    "for clf in classifiers:\n",
    "    clf.fit(X_train, y_train)\n",
    "    name = clf.__class__.__name__\n",
    "    \n",
    "    print(\"=\"*30)\n",
    "    print(name)\n",
    "    \n",
    "    print('****Results****')\n",
    "    train_predictions = clf.predict(X_test)\n",
    "    acc = accuracy_score(y_test, train_predictions)\n",
    "    print(\"Accuracy: {:.4%}\".format(acc))\n",
    "    \n",
    "    train_predictions = clf.predict_proba(X_test)\n",
    "    ll = log_loss(y_test, train_predictions)\n",
    "    print(\"Log Loss: {}\".format(ll))\n",
    "    \n",
    "    log_entry = pd.DataFrame([[name, acc*100, ll]], columns=log_cols)\n",
    "    log = log.append(log_entry)\n",
    "    \n",
    "print(\"=\"*30)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 347,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA14AAAHwCAYAAAC2Ub5FAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJzs3XmYXmV9//H3QNjB8FMQ8dModVciBAgYLAYwLpUqi6LggoIoxWoBFddSQGtbBBWjohYVERdEUVERFRdAQVECsoh1Q8H4rRapEPZgwvz+OHfkcZxkJiQnkwzv13XNNc9zlvt8zzP8kQ/f+9zP0PDwMJIkSZKk/qw10QVIkiRJ0mRn8JIkSZKknhm8JEmSJKlnBi9JkiRJ6pnBS5IkSZJ6ZvCSJEmSpJ4ZvCRJWkmSHJvkEz2Of3WS3drroSQfTXJjkh8meVKSn/V1bUnSipky0QVIkrQmSfIC4DXAY4BbgMuBf6+qC/u+dlVtPfB2F+CpwN9U1W1t26NX9jWTHAscAzyhqn64sseXpPsKO16SJI1TktcA7wb+A9gCeAjwfmCvCSjnocC1A6HrXksy6v+ITTIEHAD8EXjJil5nOWsaSuK/UyRNGna8JEkahyRTgbcCB1XV5wd2fbn9jHbOZ4EnARsAVwCvqKqr2749gHcA04CbgROr6h1JNgNOpeto3Q1cDexaVXcnuRZ4GV3oOglYJ8mtwDuB84BPVNXftPEfDLwXmA3c2sZ/T9t3LDAduBPYk66D9+FRbuFJwIOBlwNzk7y6qu4auL+Xt3P/BpgPvKiqLksyDZjbzl8LOL2qXtWu+4iqelE7fyvg18A6VbUoyfnARcBuwPbA45M8CXh9u8YfgLdX1X8N1LAX8BbgYW3/K4FNgDdW1Q4Dx70WeFJV7T3a30qS+ub/SZIkaXx2BtYHvrAc53wVeCTwQOAy4JMD+z4C/GNVbUIXgr7dtr8W+C2wOV1X7c3A8OCgVfUR4FDg+1W1cVUdM7i/dYq+TBf2AswBjkjy9IHD9gLOBDYdUdegl7RxzmjvnzlwjecCxwIvBu5HF+D+L8nawNnAdcBW7fqfXsr4ozkAOIQuPF0HXN+uez/gIODEJNu3GnYCTgNe1+5jNnAt8CXgb5M8dmDcFwEfX446JGmlsuMlSdL4PAC4oaoWjfeEqjplyevW7bkxydSqWgD8CXhckiuq6kbgxnbon4AtgYdW1S+B796LWncENq+qt7b3v0ryIWB/4Ott2/er6qz2+o6RAyTZEHgu8OKq+lOSM+mC2JJu38uA46vqkvb+l+28nem6ZK8b+KyW5/m3U5d0BZuvDLy+IMm5dJ20y4CDgVOq6httfw3UfwZd2PqXJFvThcCzl6MOSVqp7HhJkjQ+/wdstrTnoUZKsnaS45Jck+Rmuk4MwGbt93OAPYDrklzQAgvACXQh5twkv0ryxntR60OBBye5ackPXedsi4Fj5o8xxj7AIuCc9v6TwDOSbN7eTwOuGeW8acB1yxNQR/iLupI8I8nFSf7Y7mMP7vkMl1YDwMeAFww8p/aZqlp4L2uSpBVm8JIkaXy+T/dM1HifEXoB3XS+pwBT6TouAEMAVXVJVe1FNw3xLOAzbfstVfXaqnoY8CzgNUnmLGet84FfV9WmAz+bVNUeA8cML+3k5iXAxsBvkvwe+CywDvD8gWs8fCnXfshSAuptwIYD7x80yjF/rivJesDn6J6F26KqNqULgkNj1EBVXQzcRdcdewFOM5Q0wZxqKEnSOFTVgiRHAyclWQScSzct8CnA7lX1+hGnbAIspOuUbUi3EiIASdalm8Z3dhv3ZmBx2/dM4Kd0nZwl2xcvZ7k/BG5O8gbgPXQB5LHABgNTA5cqyZLnwp4BXDmw6wi6QPYeusU43pXkQrppfw+n+zx+CPwOOC7JMa32HarqIrql99+Q5CHAAuBNY5SyLrAe3aIZi5I8A3ga8OO2/yN0ncGz6RYX2RLYpKp+2vafBrwPWLQqlvuXpGWx4yVJ0jhV1bvoVvE7ii4MzAdeRdexGuk0usUhCvgJcPGI/QcA17bQdSjd80jQLcbxTbqVCL8PvL+qzl/OOhfTdctm0K0aeANdUJo6ziEOAC6vqnOr6vdLfugC1zZJplfVZ4F/Bz5F931mZwH3H7j2I4Df0C0Usl+r6xt0C3VcCVzKGM9cVdUtwGF03cAb6TpXXxrY/0Paght0Qe4CummWS3ycbuESu12SJtzQ8PBYMw0kSZLWPEk2oFsVcfuq+sVE1yPpvs2OlyRJmqxeAVxi6JK0OvAZL0mSNOm0L5seYvyLoUhSr5xqKEmSJEk9c6qhJEmSJPXM4CVJkiRJPfMZL01K06dPH542bdpElyFJkqRJ7sorr7yhqjYf6ziDlyaladOm8dWvfnWiy5AkSdIkl+S68RznVENJkiRJ6pnBS5IkSZJ65lRDTUq3zZ/PxYcfPtFlSJIkrdZmzZ070SXcZ9jxkiRJkqSeGbwkSZIkqWcGL0mSJEnqmcFLkiRJknpm8JIkSZKknhm8JEmSJKlnBi9JkiRJ6pnBS5IkSZJ6ZvCSJEmSpJ4ZvCRJkiSpZwYvSZIkSeqZwUuSJEmSembwkiRJkqSeGbzGIcmtA6/3SPKLJA9JcmyS25M8cLRjlzHeOUk2HeOY85PMHGX7gUnet7z3MB5Jjkzy0yQ/TnJFkhcvq5Z7eY2ZSd7TXq+X5JtJLk+yX5IPJ3ncyriOJEmStDqZMtEFrEmSzAHeCzytqn6TBOAG4LXAG8Y7TlXt0U+Fy5ZkCBiqqrtH2Xco8FRgp6q6OclUYO+VXUNVzQPmtbfbAetU1Yz2/ozlGSvJ2lW1eGXWJ0mSJPXB4DVOSZ4EfAjYo6quGdh1CnBgkrdX1R9HnPMi4DBgXeAHwD9V1eIk1wIzq+qGJP8KvBCYTxfiLq2qd7Qhnpvk/cCmwMFV9d22fVqSrwF/C3yqqt7Srvca4KXtmA9X1buTbAV8FTgP2BnYO8lbgJnAMHBKVZ0IvBnYvapuBqiqBcDHRvkcPgDsCGwAnFlVx7TtxwF7AouAc6vqyCTPBY4BFgMLqmp2kt2AI1udnwA2T3I58BzgI8CRVTUvydOAtwDrAdcAB1XVre2zOwV4GvA+4NOj/sEkSZKk1YhTDcdnPeCLwN5V9dMR+26lCwKHD25M8lhgP+DvWkdnMV3AGjxmJl3g2A54Nl0YGjSlqnYCjqALMEvs1MaaQRfOZibZATgIeAIwC3h5ku3a8Y8GTquq7YDNgFTV9Kp6PPDRJJsAm4wIlEvzL1U1E9gG2DXJNknuD+wDbF1V2wBva8ceDTy9qralC2V/VlXXAy8DvltVMwavnWQz4CjgKVW1PV2H7DUDp99ZVbtUlaFLkiRJawQ7XuPzJ+B7wMGMCFjNe4DLk7xzYNscYAfgkjYlcQPg+hHn7QJ8saruAEjy5RH7P99+XwpsNbD9G1X1f+2cz7dxhoEvVNVtA9ufBHwJuK6qLm7n/gp4WJL3Al8BzgU2buePx/OSHEL3386WwOOAnwB3Ah9O8hXg7HbsRcCpST4zcC/jMauNe1H77NYFvj+wf9Qpia2uQwA223DD5bicJEmS1C87XuNzN/A8YMckbx65s6puAj4F/NPA5iHgY62bM6OqHl1Vx444dWiM6y5svxfzlyF5ZEgaHmOs2wZqvRHYFjgfeCXdlMSbgduSPGxZxST5W7ppgnNaZ+srwPpVtYiuC/c5uufCvtaudShd52oaXTB9wLLGHzBEFy6XfHaPq6qDR7ufQVV1clXNrKqZU9dbb5yXkiRJkvpn8BqnqrodeCbwwiQHj3LIu4B/5J6A9C1g3yUrHia5f5KHjjjnQuBZSdZPsjHwD+Ms56ltvA3ogs5FwHfont/aMMlGdFP/vjvyxDaNb62q+hzwr8D2bdd/AicluV877n6tgzTofnShZ0GSLYBntGM3BqZW1Tl00yJntO0Pr6ofVNXRdM+vTRvn/V0M/F2SR7RxNkzyqHGeK0mSJK12DF7LoS2e8ffAUUn2GrHvBuALdM+DUVU/oev2nJvkSuAbdFPzBs+5hG4q4BV0U/HmAQvGUcqFwMeBy4HPVdW8qroMOBX4Id1CHh+uqh+Ncm6A89uCFqcCb2rbP0C3AMclSX4MXADcPqLeK4AfAVfTPdd2Udu1CXB2u88LgFe37SckuaqN9512n2Oqqj8ABwKntzEvBh4znnMlSZKk1dHQ8PB4H+1RH5Js3Fbr25AunBzSQpRWwOzp04ePnzNnosuQJElarc2aO3eiS1jjJbm0LT63TC6uMfFObl8avD7dM2GGLkmSJGmSMXhNsKp6wUTXIEmSJKlfPuMlSZIkST0zeEmSJElSzwxekiRJktQzg5ckSZIk9czgJUmSJEk9M3hJkiRJUs8MXpIkSZLUM4OXJEmSJPXM4CVJkiRJPTN4SZIkSVLPpkx0AVIfNpo2jVlz5050GZIkSRJgx0uSJEmSemfwkiRJkqSeGbwkSZIkqWcGL0mSJEnqmcFLkiRJknpm8JIkSZKknhm8JEmSJKlnBi9JkiRJ6plfoKxJ6bb587n48MMnugxJkrSamTV37kSXoPsoO16SJEmS1DODlyRJkiT1zOAlSZIkST0zeEmSJElSzwxekiRJktQzg5ckSZIk9czgJUmSJEk9M3hJkiRJUs8MXpIkSZLUM4OXJEmSJPXM4CVJkiRJPTN4SZIkSVLPpkx0AbrvSfIvwAuAxcDdwO+Ay6vqTQPHzABOr6rHJtkYeCfwFOBO4P+A11XVD1Z58ZIkSdK9YMdLq1SSnYFnAttX1TZ0Yeo4YL8Rh+4PfKq9/jDwR+CRVbU1cCCw2SopWJIkSVoJ7HhpVdsSuKGqFgJU1Q3ABUluSvKEgS7W84CnJ3k48ATghVV1dzvnV8CvJqB2SZIk6V6x46VV7VxgWpKfJ3l/kl3b9tPpulwkmQX8X1X9Atiabhri4okpV5IkSVpxBi+tUlV1K7ADcAjwB+CMJAcCnwb2TbIWXQA7fXnHTnJIknlJ5i1YuHAlVi1JkiStGKcaapVr3avzgfOTXAW8pKpOTXItsCvwHGDndvjVwLZJ1loy1XAZ454MnAwwe/r04Z7KlyRJkpabHS+tUkkeneSRA5tmANe116cDJwLXVNVvAarqGmAe8JYkQ22MRybZaxWWLUmSJK0QO15a1TYG3ptkU2AR8Eu6aYcAnwXmAv884pyX0S0n/8skt9OWk1815UqSJEkrzuClVaqqLgWeuJR9fwDWGWX7zcDLey5NkiRJ6o1TDSVJkiSpZwYvSZIkSeqZwUuSJEmSembwkiRJkqSeGbwkSZIkqWcGL0mSJEnqmcFLkiRJknpm8JIkSZKknhm8JEmSJKlnBi9JkiRJ6pnBS5IkSZJ6ZvCSJEmSpJ4ZvCRJkiSpZ1MmugCpDxtNm8asuXMnugxJkiQJsOMlSZIkSb0zeEmSJElSzwxekiRJktQzg5ckSZIk9czgJUmSJEk9M3hJkiRJUs8MXpIkSZLUM7/HS5PSbfPnc/Hhh090GZIkSVpJ1vTvaLXjJUmSJEk9M3hJkiRJUs8MXpIkSZLUM4OXJEmSJPXM4CVJkiRJPTN4SZIkSVLPDF6SJEmS1DODlyRJkiT1zOAlSZIkST0zeEmSJElSzwxekiRJktQzg5ckSZIk9czgJUmSJEk9mzLRBWjNlWQYeFdVvba9PxLYuKqOXcY5awHvBp4MDAN3As8DjgG+X1X/NXDs3sAhVbVHkge183YEFgLXAkdU1c97uDVJkiRppbLjpRWxEHh2ks2W45z9gAcD21TV44F9gJuA04H9Rxy7P3B6kiHgC8D5VfXwqnoc8GZgixW9AUmSJGlVMHhpRSwCTgZePXJHklOT7Dvw/tb2ckvgd1V1N0BV/baqbgS+CTwmyZbt+A2BpwBnAbsDf6qqDy4Zr6our6rv9nNbkiRJ0spl8NKKOgl4YZKp4zz+M8Czklye5J1JtgOoqsXA5+mmHQLsCZxXVbcA04FLV3LdkiRJ0ipj8NIKqaqbgdOAw8Z5/G+BRwNvAu4GvpVkTts9ON1w//Z+3JIckmReknkLFi5cnlMlSZKkXrm4hlaGdwOXAR8d2LaIFuzbM1rrLtlRVQuBrwJfTfK/wN7At4CLgC2TbAs8kXtC2NXAvoyhqk6mm/rI7OnTh1fsliRJkqSVx46XVlhV/ZFuCuHBA5uvBXZor/cC1gFIsn2SB7fXawHbANe1cYbbOB8DzqmqO9v53wbWS/LyJYMn2THJrn3dkyRJkrQyGby0srwTGFzd8EPArkl+CDwBuK1tfyDw5SQ/Bq6k64y9b+C804FtgU8v2dAC2T7AU5Nck+Rq4Fjgf/q5FUmSJGnlGhoedkaWJp/Z06cPHz9nztgHSpIkaY0wa+7ciS5hVEkuraqZYx1nx0uSJEmSembwkiRJkqSeGbwkSZIkqWcGL0mSJEnqmcFLkiRJknpm8JIkSZKknhm8JEmSJKlnBi9JkiRJ6pnBS5IkSZJ6ZvCSJEmSpJ4ZvCRJkiSpZwYvSZIkSerZlIkuQOrDRtOmMWvu3IkuQ5IkSQLseEmSJElS7wxekiRJktQzg5ckSZIk9czgJUmSJEk9M3hJkiRJUs8MXpIkSZLUM4OXJEmSJPXM4CVJkiRJPfMLlDUp3TZ/PhcffvhElyH9Bb/UW5Kk+y47XpIkSZLUM4OXJEmSJPXM4CVJkiRJPTN4SZIkSVLPDF6SJEmS1DODlyRJkiT1zOAlSZIkST0zeEmSJElSzwxekiRJktQzg5ckSZIk9czgJUmSJEk9M3hJkiRJUs+mTHQBq4Mki4GrgHWARcDHgHdX1d33Yqy3At+pqm8uZf+hwO1Vddpyjvt04O3t7SOAAu4ArqyqFy9vnaOMfz/gncCTgTuBPwCvA34E3FBVm67oNdp1XgncVFWfTPI44HTgbmBf4NSqetLKuI4kSZK0OjF4de6oqhkASR4IfAqYChyzvANV1dFj7P/gvSmwqr4OfL3VeD5wZFXNG3lckilVteheXOIU4L+BR1TVcJJHAI+8N7UuS1WdNPD22cCZVfVv7f24Q1eSIWDo3oRjSZIkaVUzeI1QVdcnOQS4JMmxdNMxjwN2A9YDTqqq/wJI8nrgALqOzVer6o1JTgXOrqozkxwH7EnXRTu3qo5sY95aVe9IMgP4ILAhcA3w0qq6sQWrHwC7A5sCB1fVd5dWc5KXAU8BNm41PjXJG+mCzfp04eat7diXAK8E1gW+B7yKLmDNAJ5XVcPtc/gl8MskUwaucz/grFbTFODNVXV2kk2AzwAPBtYGjm33fwLwD+3+v1pVb0jyNuAG4Fft2ouT7Ar8PQOdtdHqb2HwLOBC4AnAM+k6f5IkSdJqzeA1iqr6VZK1gAcCewELqmrHJOsBFyU5F3gMsDfwhKq6Pcn9B8do7/cBHtM6SKNN1TsN+OequqBNUTwGOKLtm1JVOyXZo21/yhhl7wzMaMFtD+AhdOFkCDgnyROBm1tNT6yqRUlOBvanm1r4o3F0j+4A9qqqW1pn8CLgbGAP4Nqqeka796lJtmjbtx7t/qvqS0l2ogtb7x4R8JZW//XA44CDqurQMWqVJEmSVhsGr6Ubar+fBmyTZN/2fipdh+gpwEer6naAqvrjiPNvpgs0H07yFbqA8mdJpgKbVtUFbdPHgM8OHPL59vtSYKtx1HtuVd04UPMz6J7Pgq4T9ii6TtWOwLwkABsA84GrxzE+dJ/J25PsQtflm5ZkM+BK4LjW4ftyVV2U5PZ2zIdGu/8xLK3+64FrquqS0U5qncpDADbbcMPluJwkSZLUL4PXKJI8DFhM9w/9Ibqu1NdHHPP3wPDSxmgdpZ2AOXRdpVfRLVwxXgvb78WM7+9028DrIeBtVfWRETW/Gjilqv51xPZHAzOSrDVG1+vFdMFz+3Z/vwXWr6r/TjKTrsN1QpKzq+o/2ran0t3/K+gC1Xgsrf5HjLjPv1BVJwMnA8yePn2pfxtJkiRpVXM5+RGSbE733NX72vNOXwdekWSdtv9RSTYCzgVemmTDtn3kVMONgalVdQ7d9MEZg/uragFwY5IlC0ocAFzAyvF14OBWJ0n+pnWmvgk8r70myQOSPKSqfka3quPRbdEKkjw6ybNGjDsVuL6FrqcCaceG7rm1jwPvArZvz33dr6rOBl4NbLcS6pckSZLWSHa8OhskuZx7lpNfEiAAPkw31e+yFkr+AOxdVV9ri2PMS3IXcA7w5oExNwG+mGR9ug7Oq0e57kuAD7bw9ivgoJVxM1V1TpLHABe3KYW3AC+oqquSvAX4ZnuG7U/AocBv2rXfRbegxh3tPo8cMfTHgS8nmQdcBvyibd+Wbqrh3cBdbcypwOfbc3FrAa9Z0fqX82OQJEmSVhtDw8POyNLkM3v69OHj58yZ6DKkvzBr7tyJLkGSJK1kSS6tqpljHedUQ0mSJEnqmcFLkiRJknpm8JIkSZKknhm8JEmSJKlnBi9JkiRJ6pnBS5IkSZJ6ZvCSJEmSpJ4ZvCRJkiSpZwYvSZIkSeqZwUuSJEmSembwkiRJkqSeGbwkSZIkqWcGL0mSJEnq2ZSJLkDqw0bTpjFr7tyJLkOSJEkC7HhJkiRJUu8MXpIkSZLUM4OXJEmSJPXM4CVJkiRJPTN4SZIkSVLPDF6SJEmS1DODlyRJkiT1zO/x0qR02/z5XHz44RNdhiRJknq2pnx3qx0vSZIkSeqZwUuSJEmSembwkiRJkqSeGbwkSZIkqWcGL0mSJEnqmcFLkiRJknpm8JIkSZKknhm8JEmSJKlnBi9JkiRJ6pnBS5IkSZJ6ZvCSJEmSpJ4ZvCRJkiSpZwYvSZIkSerZlIkuYFVJshi4iu6efw0cUFU3rYRxtwLOrqrpK2GsU4FdgQVt0ylV9Z4VHXcp19oNuKuqvjew7cXA64Gh9nNKVb2j1XV2VZ25Eq77YOA9VbVve386sDXwUeD/Ad+pqm+u6HUkSZKk1cl9JngBd1TVDIAkHwNeCfz7xJY0qtfdm4CTZO2qWrwcp+wG3Ap8r53/DOAI4GlV9T9J1gcOWN46xlJV/wMsCV0PAp5YVQ+9N2MlmVJVi1ZmfZIkSVIf7kvBa9D3gW0AkmwMfJGu27IOcFRVfbF1sr4KXAg8EShgr6q6I8kOwCnA7W0/baz1gQ8AM4FFwGuq6rwkBwJ7A2sD04F3AuvSBZuFwB5V9celFZvk+cCb6bpQX6mqN7TttwLvAp4OvDbJHe39xsANwIFV9bskhwGHtpp+AryxvV+c5EXAPwNvAo5swYiquhP40Ci1HA08C9iALrT9Y1UNj7xGVe2fZFdgbjt1GJgNPIB7OoTnAg9Mcnmr4eC278z2GY92L+e36/4d8KX2WUqSJEmrtfvcM15J1gbm0P2jHeBOYJ+q2h7YHXhnkqG275HASVW1NXAT8Jy2/aPAYVW184jhXwlQVY8Hng98rIUx6ALXC4Cd6Dptt1fVdnQh8MUDY5yQ5PL28/g2Ne/twJOBGcCOSfZux24E/LiqngD8AHgvsG9VLQmGSzp6bwS2q6ptgEOr6lrgg8CJVTWjqr7b6rt0HB/h+6pqxxacNgCeOdo12rYjgVe2TuOTgDtGjLUncM1ADQAkWWcZ9wKwaVXtWlWGLkmSJK0R7ksdrw1aZ2UruoDxjbZ9CPiPJLOBu4EAW7R9v66qy9vrS4Gtkkyl+4f/BW37x4FntNe70AUGquqnSa4DHtX2nVdVtwC3JFkAfLltv4rWfWv+Yqphkr2A86vqD+39J+k6R2cBi4HPtUMfTReevpEEuu7a79q+K4FPJjmrnbcidk/yemBD4P7A1e1eRrvGRcC7Ws2fr6rfttrGsqx7AThjtJOSHAIcArDZhhsu521JkiRJ/RkzeCVZC7hyZSweMcHuqKoZLTidTdedeg/wQmBzYIeq+lOSa4ElXaqFA+cvpuvwDNFNmxvN0FK2jxzr7oH3d7Psv8Oyxrxz4LmuIeDqUbpwAP9AF9b2BP41ydajHHM1sAPw7aVdrHXv3g/MrKr5SY7lns/qr65RVccl+QqwB3BxkqfQdRjHsqx7AbhttI1VdTJwMsDs6dOX9jeSJEmSVrkxpxpW1d3AFUkesgrq6V1VLQAOA45sU9qmAte30LU7sMyFHtpKiAuS7NI2vXBg93eWvE/yKOAhwM9WsOQfALsm2axNk3w+cMEox/0M2DzJzu366yTZugXnaVV1Ht2KhZvSPTd1C7DJwPn/CRzfFrwgyXrtua1BS0LWDe3ZuCWLZIx6jSQPr6qrqurtwDzgMeO851HvZZznSpIkSaud8T7jtSVwdZJvJfnSkp8+C+tTVf0IuALYH/gkMDPJPLrQ9NNxDHEQcFKS7/OXzy29H1g7yVV00+EOrKqFow2wHLX+jm7hi/NazZdV1RdHOe4uuiD09iRXAJfTLQqyNvCJVtOP6J7ruolueuA+7VmyJ1XVOcBJwDeTXE03tXLKiGvcRLfgxlV00wkvabuWdo0jkvy41XMH3WIl47nnpd2LJEmStEYaGh4ee0ZWW53urww85yStVmZPnz58/Jw5E12GJEmSejZr7tyxD+pRkkurauZYx42r49UC1rXAOu31JcBlK1ShJEmSJN1HjCt4JXk5cCbwX0s2seKr40mSJEnSfcJ4n/F6Jd0X1t4MUFW/AB7YV1GSJEmSNJmMN3gtbAseAJBkCktfUl2SJEmSNGC8weuCJG+m+xLipwKf5Z4vAJYkSZIkLcN4g9cbgT/QLSP+j8A5wFF9FSVJkiRJk8mUsQ/585cof6j9SJIkSZKWwzKDV5LPVNXz2hfj/tUzXVW1TW+VSZIkSdIkMVbH64j2+5l9FyJJkiRJk9VYwetsYHvgbVV1wCqoR5IkSZImnbGC17pJXgI8McmzR+6sqs/3U5YkSZIkTR5jBa9DgRcCmwLPGrFvGDB4SZIkSdIYlhm8qupC4MIk86rqI6uoJmmFbTRtGrPmzp3oMiRJkiRg7FUNn1xV3wZudKqhJEmSJN07Y0013BX4Nn89zRCcaihJkiRJ4zLWVMNj2u+DVk05kiRJkjT5jNXxAiDJ4cBHgVuAD9EtMf/Gqjq3x9okSZIkaVJYa5zHvbSqbgaeBjwQOAhoEB+vAAAgAElEQVQ4rreqJEmSJGkSGW/wGmq/9wA+WlVXDGyTJEmSJC3DeIPXpUnOpQteX0+yCXB3f2VJkiRJ0uQx3uB1MPBGYMequh1Yh266oSRJkiRpDONaXAPYGbi8qm5L8iK6xTX8dlqttm6bP5+LDz98ostYIX4BtCRJ0uQx3o7XB4Dbk2wLvB64Djitt6okSZIkaRIZb/BaVFXDwF7A3KqaC2zSX1mSJEmSNHmMd6rhLUneBLwImJ1kbbrnvCRJkiRJYxhvx2s/YCFwcFX9HghwQm9VSZIkSdIkMq6OVwtb7xp4/xt8xkuSJEmSxmVcwSvJLOC9wGOBdYG1gVuramqPtUmSJEnSpDDeqYbvA54P/ALYAHgZcFJfRUmSJEnSZDLe4EVV/RJYu6oWV9VHgd16q0qSJEmSJpHxrmp4e5J1gcuTHA/8Dtiov7IkSZIkafIYb8frALrnul4F3AZMA57TV1GSJEmSNJmMd1XD69rLO4C39FeOJEmSJE0+ywxeSa4Chpe2v6q2WekVSZIkSdIkM1bH69nAFsD8EdsfCvxPLxWt4ZLsA3weeGxV/XSU/acCZ1fVmcsY41RgV2ABsD5welWttE5jkr2Bn1fVTwa2HUm3WuUiYDHwzqo6Lcn5wJFVNW8lXHcm8OKqOizJesBXgM2A/wSeCrxrsCZJkiRpshgreJ0IvHlgqiEASTZv+57VV2FrsOcDFwL7A8euwDivq6ozk6wP/CTJaVX165VRILA3cDbwE4Akh9IFn52q6uYkU9sxK1ULb0sC3HbAOlU1o70/Y3nGSrJ2VS1emfVJkiRJfRkreG1VVVeO3FhV85Js1U9Ja64kGwN/B+wOfAk4NskQ3ZdPPxn4NTA0cPzRdOF1A+B7wD9W1cipneu337e1c+YA76D7210CvKKqFi5j+3HAnnSdrHPpunF7ArsmOYpukZQ3A7tX1c0AVbUA+Ngo9/cBYMdW75lVdUzb/hfXqKojkzwXOIaue7agqmYn2Q04Engp8Alg8ySXtxo+QuusJXka3bOE6wHXAAdV1a1JrgVOAZ5G991yn172X0SSJElaPYy1quH6y9i3wcosZJLYG/haVf0c+GOS7YF9gEcDjwdeDjxx4Pj3VdWOVTWd7vN85sC+E1oo+S3w6aq6vnW/TgX2q6rH04WsVyxj+/3b9bduz+O9raq+RxcKX9e6TdcDm1TVNeO4v3+pqpnANnTBbZvRrtGOPRp4elVtSxfK/qyqrqeb1vjdqpoxeO0kmwFHAU+pqu3pOmSvGTj9zqrapaoMXZIkSVpjjBW8Lkny8pEbkxwMXNpPSWu053NPF+bT7f1sume0FlfV/wDfHjh+9yQ/aIuYPBnYemDfkmD0IGBOkifSBbhft2AHXVdq9jK23wzcCXw4ybOB20epeYhlLKAywvOSXAb8qNX6uGVc4yLg1Pbfz9rjHB9gVhv3ohY8X0L3TOESS52SmOSQJPOSzFuwcOFyXFKSJEnq11hTDY8AvpDkhdwTtGYC69J1OdQkeQBdeJqeZJgubAwDX2CUYNO6VO8HZlbV/CTHMkqHsU2xOx/YhW6q4GiGRttYVYuS7ATMoXvm7FWtxsFjbk5yW5KHVdWvlnF/f0s3TXDHqrqxLQCy/tKuUVWHJnkC8A90X7w9Y2ljj3Iv36iq5y9l/21LO7GqTgZOBpg9ffp4w6QkSZLUu2V2vKrqf6vqiXTP21zbft5SVTtX1e/7L2+Nsi9wWlU9tKq2qqppdM90/RHYP8naSbake/4L7glZN7Rnw/YdbdAkU4An0D3r9FNgqySPaLsPAC5Y2vY27tSqOocuRC8JP7cAmwxc5j+Bk5Lcr13zfkkOGVHK/ehCz4IkWwDPaMeOeo0kD6+qH1TV0cANdF+6PR4XA3+35F6SbJjkUeM8V5IkSVotjfcLlM8Dzuu5ljXd84HjRmz7HPBY4BfAVcDP6YISVXVTkg+17dfSLYgx6IS2+MW6wLeAz1fVcJKDgM+2QHYJ8MG2iMZfbQfuD3yxddeGgFe3sT8NfCjJYXSB7wPAxnRTS/8E/Al452AxVXVFkh8BVwO/optKCF2AG+0aJyR5ZNv2LeAKuiXyl6mq/pDkQOD0tuQ8dM98/XzpZ0mSJEmrt6HhYWdkafKZPX368PFz5kx0GStk1ty5E12CJEmSxpDk0rYA3TKNtbiGJEmSJGkFGbwkSZIkqWcGL0mSJEnqmcFLkiRJknpm8JIkSZKknhm8JEmSJKlnBi9JkiRJ6pnBS5IkSZJ6ZvCSJEmSpJ4ZvCRJkiSpZwYvSZIkSeqZwUuSJEmSejZloguQ+rDRtGnMmjt3osuQJEmSADtekiRJktQ7g5ckSZIk9czgJUmSJEk9M3hJkiRJUs8MXpIkSZLUM4OXJEmSJPXM4CVJkiRJPTN4SZIkSVLP/AJlTUq3zZ/PxYcfPtFlSJJWollz5050CZJ0r9nxkiRJkqSeGbwkSZIkqWcGL0mSJEnqmcFLkiRJknpm8JIkSZKknhm8JEmSJKlnBi9JkiRJ6pnBS5IkSZJ6ZvCSJEmSpJ4ZvCRJkiSpZwYvSZIkSeqZwUuSJEmSembwkiRJkqSeTZnoApZXki2AE4FZwI3AXcDxVfWFeznescCtVfWOJG8FvlNV37wX48wAHlxV57T3BwInAAWsA/w38OKquv3e1DmO6+0JPK6qjruX460D/BvwHGAhcDtwTFV9Ncm1wMyqumEl1P3nOpNsDpwNrAscBrwJeEFV3bSi15EkSZJWJ2tU8EoyBJwFfKyqXtC2PRTYc8RxU6pq0fKOX1VHr0B5M4CZwDkD286oqle1mj4F7Ad8dAWusdTrVdWXgC+twHj/BmwJTK+qhS3g7rrCVY4wos45wE+r6iXt/XeXZ6wka1fV4pVZnyRJktSHNSp4AU8G7qqqDy7ZUFXXAe9tHaZ/ANYHNmqdlS8C/4+u43RUVX0RIMm/AC8G5gN/AC5t208Fzq6qM5PsALwL2Bi4ATiwqn6X5HzgB8DuwKbAwe39W4ENkuwC/Odg0UmmABvRdeiWhMVTgM3b9Q+qqt8sY/tzgWOAxcAC4CmjXG8Duq7Uq9p93EwXzB4EvL7d01rA++gC1a/pppqeQhfeXg78bVUtbJ/r/wKfGfkHSHIWMK19znOr6uQkawMfadcbBk6pqhOTHAYcCiwCflJV+7e/00zgw8Dx7R4uB3am6wrOrKobkryIrgu2bvt8/6mqFie5tf1dng68FrhwZI2SJEnS6mZNe8Zra+CyZezfGXhJVT0ZuBPYp6q2pwtJ70wy1ALV/sB2wLOBHUcO0qbdvRfYt6p2oAsn/z5wyJSq2gk4gm463l3A0XQdrhlVdUY7br8WKgq4P/Dltv19wGlVtQ3wSeA9Y2w/Gnh6VW0L7LmM6w3aEtgFeCawZPrhs4GtgMcDL2ufF8AjgN9U1c2jfqp/6aXtM5kJHJbkAXTdt1TV9Kp6PPd09d4IbNfu59DBQarq8hH3cMeSfUkeS9cd/LuqmkEXOF/Ydm8E/LiqnlBVhi5JkiStEda0jtdfSHISXbi4CzgJ+EZV/bHtHgL+I8ls4G4gwBbAk4AvLHnWKslo0/MeDUwHvpEEYG3gdwP7P99+X0oXZJbmjNaBGmr1vY4uBO1MF4IAPk7X+WEZ2y8CTk3ymYFrj+Wsqrob+EmbNgjdZ/XZtv33Sc4b51iDDkuyT3s9DXgk8DPgYUneC3wFOLftvxL4ZOuSnbUc15gD7ABc0j7/DYDr277FwOdGOynJIcAhAJttuOFyXE6SJEnq15rW8boa2H7Jm6p6Jd0/0jdvm24bOPaFbfsOrWvyv3TT46CbDrcsQ8DVrRMzo6oeX1VPG9i/sP1ezDjCa1UN03W7Zi/lkKXVM9zOPxQ4ii7oXN66TGNZOPB6aMTvkX4JPCTJJssaMMludNMcd27dtx8B61fVjcC2wPnAK+mmEUI39fMkuhB1aZtyOR5DdM/xLfn8H11Vx7Z9dy7tua6qOrmqZlbVzKnrrTfOS0mSJEn9W9OC17eB9ZO8YmDb0lobU4Hrq+pPSXYHHtq2fwfYJ8kGLWg8a5RzfwZsnmRn6KYeJtl6jNpuAZYVXHYBrmmvv0c33RG6gHjhsrYneXhV/aAt/nEDXQAb63qjuRB4TpK1WhdsN4DW/fsI8J4k67Zrbtmesxo0Fbixqm5P8hi6lSVJshmwVlV9DvhXYPv2PNm0qjoPeD3d83Abj7PObwH7JnlgG//+7fk3SZIkaY20Rk01rKrhJHsDJyZ5Pd0CFLcBb6Cbjjbok8CXk8wDLgd+2sa4LMkZbdt1jLKSXlXdlWRfuiAyle5zejddx21pzgPe2J7pWrK4xn5t8Yu1gN8CB7bthwGnJHldu4eDxth+QpJH0nWCvgVcAfxmlOuN5XN0HcIfAz+nW7RiQdt3FPA2uqmJd9J9riNXefwacGiSK+nC6cVte4CPtrAF3bLwawOfaJ/fEHBiVd3Upg4uU1X9JMlRwLltzD/RddKuG+d9SpIkSauVoeHhsWbdaTJJsnFV3dqmK/6QbgGL3090XSvb7OnTh4+fM2eiy5AkrUSz5s6d6BIk6a8kubSqZo513BrV8dJKcXaSTemWaf+3yRi6JEmSpNWNwes+pqp2m+gaJEmSpPuaNW1xDUmSJEla4xi8JEmSJKlnBi9JkiRJ6pnBS5IkSZJ6ZvCSJEmSpJ4ZvCRJkiSpZwYvSZIkSeqZwUuSJEmSembwkiRJkqSeGbwkSZIkqWdTJroAqQ8bTZvGrLlzJ7oMSZIkCbDjJUmSJEm9M3hJkiRJUs8MXpIkSZLUM4OXJEmSJPXM4CVJkiRJPTN4SZIkSVLPDF6SJEmS1DODlyRJkiT1zC9Q1qR02/z5XHz44RNdhiRpOc2aO3eiS5CkXtjxkiRJkqSeGbwkSZIkqWcGL0mSJEnqmcFLkiRJknpm8JIkSZKknhm8JEmSJKlnBi9JkiRJ6pnBS5IkSZJ6ZvCSJEmSpJ4ZvCRJkiSpZwYvSZIkSeqZwUuSJEmSejZlogvQypVkC+BEYBZwI3AXcHxVfaHHa84EXlxVh93L868FLq2q57T3+wLPrKoDkxwInAAUsA7w3+1at6+M2iVJkqRVwY7XJJJkCDgL+E5VPayqdgD2B/6mz+tW1bx7G7oGzEyy9VL2nVFVM6pqa7ogud8KXkuSJElapex4TS5PBu6qqg8u2VBV1wHvTbIV8HFgo7brVVX1vSS7AUdW1TMBkrwPmFdVpyY5DtgTWAScW1VHJnkucAywGFhQVbMHx0iyE/BuYAPgDuCgqvpZ61ztCWwIPBz4QlW9fqD2dwBvBl64tJtLMqXVf+O9/oQkSZKkCWDHa3LZGrhsKfuuB55aVdvTdYzes6yBktwf2AfYuqq2Ad7Wdh0NPL2qtqULUiP9FJhdVdu1Y/9jYN+Mdu3HA/slmTaw7zPA9kkeMcqY+yW5nG664f2BLy+rdkmSJGl1Y/CaxJKclOSKJJfQPR/1oSRXAZ8FHjfG6TcDdwIfTvJsYMkzVRcBpyZ5ObD2KOdNBT6b5Md0z5oNTh/8VlUtqKo7gZ8ADx3Yt5juWa43jTLmGVU1A3gQcBXwuqXc7yFJ5iWZt2DhwjFuT5IkSVp1DF6Ty9XA9kveVNUrgTnA5sCrgf8FtgVmAuu2wxbxl/8drN/OXQTsBHwO2Bv4Wtt+KHAUMA24PMkDRtTwb8B5VTUdeNaS8ZrBNLSYv57q+nFgNvCQ0W6uqobpul2zl7L/5KqaWVUzp6633miHSJIkSRPC4DW5fBtYP8krBrZt2H5PBX5XVXcDB3BPt+o64HFJ1ksylS6okWRjYGpVnQMcQTdNkCQPr6ofVNXRwA10AWzQVLopgQAHLk/xVfUnui7ZEcs4bBfgmuUZV5IkSZpoLq4xiVTVcJK9gROTvB74A3Ab8Aa6Z78+1xbHOK9tp6rmJ/kMcCXwC+BHbbhNgC8mWR8YouuYAZyQ5JFt27eAK4BdB8o4HvhYktfQBcHl9RG6jtqg/ZLsQvc/Cn7LcgY6SZIkaaINDQ8PT3QN0ko3e/r04ePnzJnoMiRJy2nW3LkTXYIkLZckl1bVzLGOc6qhJEmSJPXM4CVJkiRJPTN4SZIkSVLPDF6SJEmS1DODlyRJkiT1zOAlSZIkST0zeEmS/n97dx5mSVWnefxbVgHKvuig/CgpmMFGKGSxcEobbbZBpHFrV6QRcKFxhgF9tJXRdhtkWh1c0nHpQVRcUEFkU1uwh0aFbqEtqpACN3wELF5okK1AQFTI+SNOwiXNzCqEyJtkfT/PU0/dWM+JuJGR981z4lxJktQzg5ckSZIk9czgJUmSJEk9M3hJkiRJUs8MXpIkSZLUM4OXJEmSJPVs3rArIPVhvfnzWTwyMuxqSJIkSYAtXpIkSZLUO4OXJEmSJPXM4CVJkiRJPTN4SZIkSVLPDF6SJEmS1DODlyRJkiT1zOAlSZIkST0zeEmSJElSz/wCZc1Kd65YwUVHHz3sakiSJKkHi0dGhl2Fh8wWL0mSJEnqmcFLkiRJknpm8JIkSZKknhm8JEmSJKlnBi9JkiRJ6pnBS5IkSZJ6ZvCSJEmSpJ4ZvCRJkiSpZwYvSZIkSeqZwUuSJEmSembwkiRJkqSeGbwkSZIkqWcGL0mSJEnq2YwPXlX1mwnmHVFVr56Gsq+uquXt34+r6n1VtU5btkVVnfYIlPGCqjrmIW7zj1W18cMte9w+F1TVqyaYP1JVqaqHda20c/n4P2G7R/xYJUmSpOk2b9gV+FMk+Yc+919Vc4A5bXLPJDdV1frACe3fIUmuA176MMuZl+Rs4OyHsl2S/R9OuZNYALwK+PLYjBa2XgysAJ4DfLeHcqfU07FKkiRJ0+pRGbyq6j3Ab5IcX1XfBS4G9gQ2Bl6b5IKqmgu8H9gDWAf4RJL/2wLUWcAmwFrA3yU5q6oWAN8GzgeeCbxosMwkv6mqI4AVVbUpsCHwzSQLq2oH4HPA2nStiC9JcmVrlXsLMApcluTgqjoJuAXYBVhaVcuBRUmObMvuBrYDtgIOAw5p9bk4yaHt+K8GFgHrtzpfCDwLCPDCJHdX1euBw1udfgEcnOSuVsbtbfsnAm9Nclo7V0+tqkuBzyf5SDunlwOnAAfSglc7/08Gtmn/fzTJx9qyM4H5wGOBkSQnjHvvjgVuSjLSpo8DbgC+1srZkO66fEN7H8eO9W7gVGBLYC5wbJJTkCRJkh4FZnxXw9U0L8kzgDcC727zXgusTLIbsBvw+qraGvgt8OIku9IFiw+1Fi6APwO+kGSXJNeMLyTJ7cBVwLbjFh1BFzJ2pgsJ17Yw9g5gryQ7AUcPrP8UYJ8kb57gWDYB9gLeBHwD+AiwA7BjVe08wfrb0oXKHYDbgJe0+acn2a2V/ZN2PsY8CdgdOIAucAEcA1yQZOcWuqALW18BzgAOqKq1BvaxHfBc4BnAuweWvSbJ09t5OKqqNhtX38/QhcmxFrVXAifTtbad287hTsCl47bbD7guyU5JFgLnTHAuJEmSpBlptgSv09v/l9B1mQPYF3h1a8G5GNiMLqTMAf5XVV0G/D+ggM3bNtckuWgVZc2ZYN4PgLdX1duArZLcTReeTktyE0CSWwbW/1qSeyfZ/zeSjALLgRuSLE9yH3DFwLENuirJWEgZPP6FVXVBa1E7iC68jTkzyX1JfswDx/4gVbU2sH9b93a6c7jvwCrfSnJPO74bB/ZzVFX9CLiIruXrQSE1ydXAzVW1S9vfsiQ3Az8EDmutaTsmuWNclZYD+1TVB6rq2UlWTlDnw6tqSVUtWXnPPRMdliRJkjQUsyV4jX3KvpcHuk/OAf57a8HZOcnWSb5DF0KeADy9ta7cQNctDuDOqQqpqg3ogs3PB+cn+TLwArrucOdW1V6t/NFJdjVVOWPHct/A67HpibqGDq4zePwnAUcm2RF4Lw8c4/htJgqS0LUwbQQsb939dqdrAZu03KraA9gHeGZraVs2rtwxJwKH0nWl/CxAku/TPUcW4IvjB09J8nPg6XQB7O+r6l3jd5rkhCSLkizaaJ11JjksSZIkafrNluA1kXOBN4x1gauqp1TVenRh4sYkv6+qPemepVql9mzYJ+lagG4dt2wb4JftOaezgacB5wEvH+tq154Lm04bANe34z9oNda/o20z5kDgdUkWJFkAbA3sW1XrTrGPjYBb27Nk2wGLJ1nvDLpgtxvd+0RVbUX3vnyarjviroMbVNUWwF1JvgQcP365JEmSNJM9GgbXWLeqrh2Y/vBqbnciXevU0vYM16/pBsw4GfhGVS2he47op6vYz/lt+8fQBYZjJ1jnFcBfV9XvgX8H/meSW9rAEd+rqnvpWn8OXc26PxLeSdc98Bq6VqINpl6dy4A/tG6Cp9I9v/U3YwuT3FlVFwLPn2If5wBHtG6cP6PrbvhHkvyuqs4HbhvocrkH8LftHP4GGP91ATsC/7uq7gN+D7xhFccjSZIkzRhzRkcn6w0n9aMNqrEUeFmSK/so4zkLF45+cO+9+9i1JEmShmzxyMiwq3C/qrokyaJVrTebuxpqBqqq7emGtz+vr9AlSZIkzTSPhq6GmkXaSIrbDLsekiRJ0nSyxUuSJEmSembwkiRJkqSeGbwkSZIkqWcGL0mSJEnqmcFLkiRJknpm8JIkSZKknhm8JEmSJKlnBi9JkiRJ6pnBS5IkSZJ6ZvCSJEmSpJ7NG3YFpD6sN38+i0dGhl0NSZIkCbDFS5IkSZJ6Z/CSJEmSpJ4ZvCRJkiSpZwYvSZIkSeqZwUuSJEmSembwkiRJkqSeGbwkSZIkqWcGL0mSJEnqmV+grFnpzhUruOjoo1e5nl+yLEmSpOlgi5ckSZIk9czgJUmSJEk9M3hJkiRJUs8MXpIkSZLUM4OXJEmSJPXM4CVJkiRJPTN4SZIkSVLPDF6SJEmS1DODlyRJkiT1zOAlSZIkST0zeEmSJElSzwxekiRJktSzXoNXVW1ZVWdV1ZVV9cuq+nhVrfMI7HePqvrmQ9xmQVW9amB6UVV9bBXbXF1Vy9u/H1fV+8bqX1VbVNVpf9oRPKiMF1TVMQ9xm3+sqo0fbtnj9vmg8zMwf6SqUlUP61pp5/Lxf8J2j/ixSpIkSdOtt+BVVXOA04Ezk2wLbAs8Dvhgj2XOm2LxAuD+YJFkSZKjVmO3eybZEXgGsA1wQtv+uiQvfRjVparmJTk7yfsfynZJ9k9y28MpewILGDg/AC1svRhYATznES5vtfR0rJIkSdK0miqoPFx7Ab9N8jmAJPdW1ZuAa6rqSmC7JEcCtNar45N8t6o+BexGF9JOS/Luts5+wEeBm4ClY4VU1XuALeiCw01V9Xbgi8B6bZUjk/wr8H7gqVV1KfB5YBnwliQHVNX6wP8BFgGjwHuTfH3wYJL8pqqOAFZU1abAhsA3kyysqh2AzwFr04XZlyS5sqpeDbyl7fOyJAdX1UnALcAuwNKqWg4sSnJkW3Y3sB2wFXAYcAjwTODiJIe2Y7661XV94NvAhcCzgAAvTHJ3Vb0eOLzV6RfAwUnuamXc3rZ/IvDWJKeNPz9JPgLsCVwOnAIcCHx34Jw/mS6IPhn4aJKPtWVnAvOBxwIjSU4YPI9VdSxwU5KRNn0ccAPwtVbOhnTX5RuSXDBwrHcDpwJbAnOBY5OcgiRJkvQo0GdXwx2ASwZnJLkduJqpA987kiwCngb8RVU9raoeC3waeD7wbLrAMOjpdIHjVcCNwH9JsivwCmCsO+ExwAVJdm6hYtA7gZVJdkzyNOCfJ6pYq/9VdK13g46gCxk704WEa1sYewewV5KdgKMH1n8KsE+SN09QzCZ0ofVNwDeAj9Cdyx2raucJ1t8W+ESSHYDbgJe0+acn2a2V/RPgtQPbPAnYHTiALnDBxOfnQOArwBnAAVW11sA+tgOeS9cS+O6BZa9J8vR2Ho6qqs3G1fczdGFyrEXtlcDJdK1t57ZzuBNw6bjt9gOuS7JTkoXAOROcC0mSJGlG6jN4zaFr6Zlo/lReXlVL6VqkdgC2p/uQf1WSK5OMAl8at83ZSe5ur9cCPt1akr7Wtl+VfYBPjE0kuXWKdSeq/w+At1fV24CtWl32omuxu6nt85aB9b+W5N5J9v+NdozLgRuSLE9yH3AFXaveeFclGQsplwyss7CqLmjn4SC6cznmzCT3JfkxsPlElaiqtYH927q3AxcD+w6s8q0k97Tju3FgP0dV1Y+Ai+havh4UUpNcDdxcVbu0/S1LcjPwQ+Cw1pq2Y5I7xlVpObBPVX2gqp6dZOUEdT68qpZU1ZKV99wz0WFJkiRJQ9Fn8LqCrtXjflW1Id0H9JvHlf3Ytnxruq55e7eWp2+NLWPiEDfmzoHXb6LrurZTK3/t1ajrZCHxQapqA7pg8/PB+Um+DLyArjvcuVW11yr2eeck8wHGEsN9A6/HpidqKRxc596BdU6i62a5I/BeHjiP47eZLAjvB2wELG/d/XanawGbtNyq2oMuxD6ztbQtG1fumBOBQ+m6Un4WIMn36Z4jC/DF1k3zfkl+TteyuRz4+6p61/idJjkhyaIkizZa52GP4SJJkiQ9YvoMXucB6459gK6qucCHgI/TddfbuaoeU1Xz6bqrQfd8z53AyqraHHhem/9TYOuq+o9tejAAjLcRcH1rJTqY7nkggDuADSbZ5jvAkWMTVbXJ+BXac2CfpGsBunXcsm2AX7bnnM6m6yZ5Hl3r3WZtnU2nqHMfNgCub10AD1qN9cefnwOB1yVZkGQBsDWwb1WtO8U+NgJubc+SbQcsnmS9M+iC3W7AuQBVtRVwY5JP03VH3HVwg6raArgryZeA48cvlyRJkmay3oJX6y73YuClbTCNm4H7khwH/Atd+FpO9yF6advmR3StJFfQtYT8S5v/W7qBIr5VVRcC10xR9CeBQ6rqIrpnqcZaly4D/lBVP2qDfAx6H7BJVV3eusntObDs/IBK+7EAAAjUSURBVKq6HPg34FfA30xQ5iuAy9vAFNsBX0hyBXAc8L22zw9PUec+vJOue+A/0QXXVRk8P++ge37rW2MLk9xJN4jH86fYxzl0LV+XAcfSdTf8I0l+B5wPnDrQ5XIP4NKqWkb3nNrIuM12BP6tneN30L1nkiRJ0qPCnNHRVfawe0RU1bPoBmr4qySXrGp9zV5tUI2lwMuSXNlHGc9ZuHD0g3vvvcr1Fo+Mz3eSJEnS6quqS9rggFPqczj5B2lDum81XeVpZqqq7YFvAmf0FbokSZKkmWbagpcE0EZS3GbY9ZAkSZKmU5+Da0iSJEmSMHhJkiRJUu8MXpIkSZLUM4OXJEmSJPXM4CVJkiRJPTN4SZIkSVLPDF6SJEmS1DODlyRJkiT1zOAlSZIkST0zeEmSJElSz+YNuwJSH9abP5/FIyPDroYkSZIE2OIlSZIkSb0zeEmSJElSzwxekiRJktQzg5ckSZIk9WzO6OjosOsgPeKq6g7gZ8Ouh4bq8cBNw66EhsprQF4D8hrQdFwDWyV5wqpWclRDzVY/S7Jo2JXQ8FTVEq+BNZvXgLwG5DWgmXQN2NVQkiRJknpm8JIkSZKknhm8NFudMOwKaOi8BuQ1IK8BeQ1oxlwDDq4hSZIkST2zxUuSJEmSeuaohpp1qmo/YASYC5yY5P1DrpJ6VlXzgS8ATwTuA05IMlJVmwKnAAuAq4GXJ7l1WPVUv6pqLrAESJIDqmpr4KvApsBS4OAkvxtmHdWfqtoYOBFYCIwCr6H7WhHvAWuIqnoT8Dq69385cBjwJLwPzFpV9VngAODGJAvbvAl/91fVHLrPh/sDdwGHJlk6nfW1xUuzSvvg9QngecD2wIFVtf1wa6Vp8AfgzUmeCiwG/lt7348BzkuyLXBem9bsdTTwk4HpDwAfae//rcBrh1IrTZcR4Jwk2wE70V0L3gPWEFVVwFHAovYBfC7wSrwPzHYnAfuNmzfZz/3zgG3bv8OBT01THe9n8NJs8wzgF0l+2f6i9VXghUOuk3qW5Pqxv1oluYPuA1fRvfefb6t9HnjRcGqovlXVlsBf0rV40P6yuRdwWlvF938Wq6oNgecAnwFI8rskt+E9YE0zD3hcVc0D1gWux/vArJbk+8At42ZP9nP/QuALSUaTXARsXFVPmp6adgxemm0KWDEwfW2bpzVEVS0AdgEuBjZPcj104Qz4D0Osmvr1UeCtdF1NATYDbkvyhzbtvWB22wb4NfC5qlpWVSdW1Xp4D1hjJAlwPPArusC1ErgE7wNrosl+7of+GdHgpdlmzgTzHLpzDVFV6wNfB96Y5PZh10fTo6rG+vdfMjDbe8GaZR6wK/CpJLsAd2K3wjVKVW1C16KxNbAFsB5d17LxvA+suYb+e8HgpdnmWmD+wPSWwHVDqoumUVWtRRe6Tk5yept9w1g3gvb/jcOqn3r158ALqupquu7Fe9G1gG3cuhyB94LZ7lrg2iQXt+nT6IKY94A1xz7AVUl+neT3wOnAs/A+sCaa7Od+6J8RDV6abX4IbFtVW1fV2nQP1p495DqpZ+15ns8AP0ny4YFFZwOHtNeHAGdNd93UvyT/I8mWSRbQ/cz/c5KDgPOBl7bVfP9nsST/Dqyoqj9rs/YGfoz3gDXJr4DFVbVu+50wdg14H1jzTPZzfzbw6qqaU1WLgZVjXRKni1+grFmnqvan+2v3XOCzSY4bcpXUs6raHbiAbvjgsWd83k73nNepwJPpfim/LMn4h3A1i1TVHsBb2nDy2/DAMNLLgL9Ocs8w66f+VNXOdIOrrA38km4o8cfgPWCNUVXvBV5BN9LtMrqh5QvvA7NWVX0F2AN4PHAD8G7gTCb4uW+B/ON0oyDeBRyWZMl01tfgJUmSJEk9s6uhJEmSJPXM4CVJkiRJPTN4SZIkSVLPDF6SJEmS1DODlyRJkiT1zOAlSdIMVlUvrqrRqtpu2HV5qKrqCVV1YVVdXlUvGph/VlVtMcy6SdJ0M3hJkjSzHQhcSPfl0L2pqrk97PZA4PPAM4G/beU8H1ia5LoeypOkGWvesCsgSZImVlXrA38O7AmcDbxnYNlbgYPpvjT820mOqar/BPwD8ATgXuBlwHzal0q37T4OLElyUlVdDXwW2Bf4eFVtABxO9yXEvwAOTnJXVW3e9rtNK/4NwPOAm5KMtP0eB9yQ5GMDh/B74HHAOsB9VTUPeCPw/EfsJEnSo4QtXpIkzVwvAs5J8nPglqraFaCqnteW/eckOwEfbOufDHyizXsWcP1qlPHbJLsn+SpwepLd2vY/AV7b1vkY8L02f1fgCuAzwCGtPo+ha5E7edy+vww8FziHLjT+V+ALSe56aKdBkh79bPGSJGnmOhD4aHv91Ta9FNgH+NxYgElyS2utqiRntHm/BaiqVZVxysDrhVX1PmBjYH3g3DZ/L+DVbb/3AiuBlVV1c1XtAmwOLEty8+COk6wE/rLVYxPgbcBfVdWngU2ADyX5weqfDkl69DJ4SZI0A1XVZnSBZ2FVjQJzgdHWxXAOMDpukzmT7OoPPLiHy2PHLb9z4PVJwIuS/KiqDgX2WEU1TwQOBZ5I12VxKu8CjqMLj5fQtYadRdeNUpJmPbsaSpI0M72UrlveVkkWJJkPXAXsDnwHeE1VrQtQVZsmuR24dmz0wKpapy2/Bti+TW8E7D1FmRsA11fVWsBBA/PPo3uui6qaW1UbtvlnAPsBu/FA69gfqaptgS2SfA9Yl+65tFH+OARK0qxl8JIkaWY6kC7YDPo68Kok59ANtrGkqi4F3tKWHwwcVVWXAf8KPDHJCuBU4DK6Z7CWTVHmO4GLgX8Cfjow/2hgz6paTtdatQNAkt8B5wOnti6IkzkO+Lv2+it0rWQXAcdPsY0kzSpzRkfH91SQJElatTaoxlLgZUmuHHZ9JGkms8VLkiQ9ZFW1Pd2Q8+cZuiRp1WzxkiRJkqSe2eIlSZIkST0zeEmSJElSzwxekiRJktQzg5ckSZIk9czgJUmSJEk9M3hJkiRJUs/+P4buza2c6uIZAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1c4c50a588>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(12,8))\n",
    "sns.set_color_codes(\"muted\")\n",
    "sns.barplot(x='Accuracy', y='Classifier', data=log, color=\"r\")\n",
    "plt.xlabel('Accuracy %')\n",
    "plt.title('Classifier Accuracy')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 348,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Classifier</th>\n",
       "      <th>Accuracy</th>\n",
       "      <th>Log Loss</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>KNeighborsClassifier</td>\n",
       "      <td>91.414141</td>\n",
       "      <td>1.568838</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>SVC</td>\n",
       "      <td>80.808081</td>\n",
       "      <td>4.309221</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NuSVC</td>\n",
       "      <td>97.979798</td>\n",
       "      <td>2.274299</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>DecisionTreeClassifier</td>\n",
       "      <td>25.757576</td>\n",
       "      <td>6.085389</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>RandomForestClassifier</td>\n",
       "      <td>98.989899</td>\n",
       "      <td>0.737972</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>AdaBoostClassifier</td>\n",
       "      <td>7.575758</td>\n",
       "      <td>4.595045</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>GradientBoostingClassifier</td>\n",
       "      <td>58.080808</td>\n",
       "      <td>2.801577</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>GaussianNB</td>\n",
       "      <td>57.070707</td>\n",
       "      <td>14.827252</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>LinearDiscriminantAnalysis</td>\n",
       "      <td>98.484848</td>\n",
       "      <td>0.987728</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>QuadraticDiscriminantAnalysis</td>\n",
       "      <td>1.515152</td>\n",
       "      <td>34.015462</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                      Classifier   Accuracy   Log Loss\n",
       "0           KNeighborsClassifier  91.414141   1.568838\n",
       "0                            SVC  80.808081   4.309221\n",
       "0                          NuSVC  97.979798   2.274299\n",
       "0         DecisionTreeClassifier  25.757576   6.085389\n",
       "0         RandomForestClassifier  98.989899   0.737972\n",
       "0             AdaBoostClassifier   7.575758   4.595045\n",
       "0     GradientBoostingClassifier  58.080808   2.801577\n",
       "0                     GaussianNB  57.070707  14.827252\n",
       "0     LinearDiscriminantAnalysis  98.484848   0.987728\n",
       "0  QuadraticDiscriminantAnalysis   1.515152  34.015462"
      ]
     },
     "execution_count": 348,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "log"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 346,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA14AAAHwCAYAAAC2Ub5FAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJzs3Xm8X9O9//HXkSBIxFVD+UhpVbWSEASlKTHfpsZWSymlWtWr109rrKto67ZSQmtoe810UGqsqVVD1NCqhKBUB0rlUy0xJCGEJOf3x16nvj09w/ckZ+ckx+v5eJzH+X7X3nutz/4eDw9va+31bWltbUWSJEmSVJ8l+roASZIkServDF6SJEmSVDODlyRJkiTVzOAlSZIkSTUzeEmSJElSzQxekiRJklQzg5ckSb0sIk6MiB/W2P8jETG2vG6JiAsj4sWI+G1EfDAi/lDX2JKk+TOwrwuQJGlxFBF7A18C3gvMBKYA/5uZd9U9dmYOb3g7BtgeWCMzXylt6/bWWBFxETA1M4/rrT47GGMs8MPMXKOuMSSprznjJUlSD0XEl4BvA98AVgXeAXwX2LUPylkTeLIhdM23iPB/yEpSTfwXrCRJPRARQ4GvAQdk5lUNh64rPx1d81Pgg8AywIPA5zPzkXJsHHAqMAyYAZyemadGxErARVQzWvOAR4CtMnNeRDwJfIYqdJ0NLBkRLwMTgNtpmD2KiNWBM4EtgZdL/2eUYycCI4DXgF2oZvDO68FnsQXwHeA9wB+B/5eZ95Rj7wQuBjYE7gX+AAzNzE8223/pZ2ip/0PALOBc4Bvlc3g3cD4wCngDuDUz94yIFuA0YB9gaeApYO/M/F1Pxpak3uSMlyRJPbM5MAi4ugfX3ASsA6wC3A/8qOHY+cDnMnMIVQi6rbQfDkwFVqaaVTsWaG3sNDPPBw4Gfp2ZgzPzhMbjEbEEVRh8EAhgW+CwiNix4bRdgSuAFdrV1aWIWBG4ATgDeBtV0LkhIt5WTvkx8Nty7ERg32b7budMYCjwLmArYD/ggHLs68DNwH8Aa5RzAXagCprvobqvPYHn53N8SeoVznhJktQzbwOmZeacZi/IzAvaXpdZphcjYmhmTqeaqVkvIh7MzBeBF8upbwCrAWtm5p+BO+ej1k2AlTPza+X9ExFxLrAX8IvS9uvMvKa8frUHfX8Y+FNm/qC8vzQiDgV2jojbytjbZubrwF0R8bOeFh8RA6hC04aZOROYGRETqELc+VSf0ZrA6pk5FWh7vu4NYAjV83e/zczf93RsSeptznhJktQzzwMrNfs8VEQMiIiTI+LxiJgBPFkOrVR+fxQYBzwVEXdExOal/RTgz8DNEfFERBwzH7WuCaweES+1/VDNnK3acM7T89EvwOpUS/gaPUU1s7Y68EJmzlrAcVYClmo3TtsYAEcBLcBvy06PnwbIzNuAs6iWYf4jIs6JiOXnY3xJ6jUGL0mSeubXVM9E7dbk+XtTLefbjmrJ3FqlvQUgM+/LzF2pliFeA1xe2mdm5uGZ+S5gZ+BLEbFtD2t9GvhLZq7Q8DMkM8c1nNPa2cXd+BtVsGv0DiCBZ4AVI2LZhmPD5mOMabw5q9V+DDLz75n52cxcHfgc8N3y3BeZeUZmbgwMp1pyeOR8jC9JvcalhpIk9UBmTo+I44GzI2IO1TNGb1AFq60z86h2lwwBZlPNlC1LtRMiABGxFPAx4PrS7wxgbjm2E/AY8DjVphtz2471wG+BGRFxNNWzWK8D7wOWycz7etDPgIgY1PB+HnAjcGbZVv9yqpm79cq9TIuIScCJEXEcsDFVeOxw85E27caA6nO7HPjfiNgPWJFqA5BTy/kfo1oqOZVqiWYrMDciNqH6n8v3A69QBeWefnaS1Kuc8ZIkqYcy8zSqAHAc8BzVzNIXqGas2ruEanlcAo8Cv2l3fF/gyRK6Dgbadv1bB7iFaifCXwPfzcyJPaxzLlXgGQX8hWoG6TyqmbeeOIbq+a+2n9sy83lgJ6pNQJ6nWva3U2ZOK9fsQ7URyfPAScBlVEGqM9FujFeBtYH/pgpPT1A9w/VjoO2ZuU2Ae8uOjj+j2lXxL8DyVLsfvkj12T9PCWuS1FdaWlvnd4WBJElScyLiMuCx9jsvStJbhUsNJUlSryvL/V6gmmnbgeo5t5P7tChJ6kMGL0mSVIe3A1dRbb8/lepLox/o25Ikqe+41FCSJEmSaubmGpIkSZJUM4OXJEmSJNXMZ7zUL40YMaJ12LD5+a5OSZIkqXkPPfTQtMxcubvzDF7ql4YNG8ZNN93U12VIkiSpn4uIp5o5z6WGkiRJklQzg5ckSZIk1cylhuqXps58mqMnHr7Qxx0/dsJCH1OSJEmLPme8JEmSJKlmBi9JkiRJqpnBS5IkSZJqZvCSJEmSpJoZvCRJkiSpZgYvSZIkSaqZwUuSJEmSambwkiRJkqSaGbwkSZIkqWYGL0mSJEmqmcFLkiRJkmpm8JIkSZKkmhm8JEmSJKlmBq8mRMTLDa/HRcSfIuIdEXFiRMyKiFU6OreL/m6MiBW6OWdiRIzuoH3/iDirp/fQjIg4IiIei4jfRcSDEbFfV7XM5xijI+KM8nrpiLglIqZExJ4RcV5ErNcb40iSJEmLkoF9XcDiJCK2Bc4EdsjMv0YEwDTgcODoZvvJzHH1VNi1iGgBWjJzXgfHDga2BzbNzBkRMRTYrbdryMxJwKTydkNgycwcVd5f1pO+ImJAZs7tzfokSZKkOhi8mhQRHwTOBcZl5uMNhy4A9o+I8Zn5QrtrPgkcCiwF3Av8V2bOjYgngdGZOS0ivgLsAzxNFeImZ+appYuPRcR3gRWAAzPzztI+LCJ+DrwT+HFmfrWM9yXg0+Wc8zLz2xGxFnATcDuwObBbRHwVGA20Ahdk5unAscDWmTkDIDOnAxd38Dl8D9gEWAa4IjNPKO0nA7sAc4CbM/OIiPgYcAIwF5iemVtGxFjgiFLnD4GVI2IK8FHgfOCIzJwUETsAXwWWBh4HDsjMl8tndwGwA3AW8JMO/2CSJEnSIsSlhs1ZGrgW2C0zH2t37GWqIPD/Ghsj4n3AnsAHyozOXKqA1XjOaKrAsSHwEaow1GhgZm4KHEYVYNpsWvoaRRXORkfExsABwGbA+4HPRsSG5fx1gUsyc0NgJSAyc0RmjgQujIghwJB2gbIz/5OZo4H1ga0iYv2IWBHYHRiemesDJ5Vzjwd2zMwNqELZP2Xms8BngDszc1Tj2BGxEnAcsF1mbkQ1Q/alhstfy8wxmWnokiRJ0mLBGa/mvAHcAxxIu4BVnAFMiYgJDW3bAhsD95UlicsAz7a7bgxwbWa+ChAR17U7flX5PRlYq6H9l5n5fLnmqtJPK3B1Zr7S0P5B4GfAU5n5m3LtE8C7IuJM4AbgZmBwub4ZH4+Ig6j+2VkNWA94FHgNOC8ibgCuL+feDVwUEZc33Esz3l/6vbt8dksBv2443uGSxFLXQQDLrbRsD4aTJEmS6uWMV3PmAR8HNomIY9sfzMyXgB8D/9XQ3AJcXGZzRmXmupl5YrtLW7oZd3b5PZd/DcntQ1JrN3290lDri8AGwETgEKoliTOAVyLiXV0VExHvpFomuG2Z2boBGJSZc6hm4a6kei7s52Wsg6lmroZRBdO3ddV/gxaqcNn22a2XmQd2dD+NMvOczBydmaOXHrJ0k0NJkiRJ9TN4NSkzZwE7AftExIEdnHIa8DneDEi3Anu07XgYEStGxJrtrrkL2DkiBkXEYODDTZazfelvGaqgczfwK6rnt5aNiOWolv7d2f7Csoxvicy8EvgKsFE59E3g7IhYvpy3fJlBarQ8VeiZHhGrAh8q5w4GhmbmjVTLIkeV9rUz897MPJ7q+bVhTd7fb4APRMS7Sz/LRsR7mrxWkiRJWuQYvHqgbJ7xn8BxEbFru2PTgKupngcjMx+lmu25OSIeAn5JtTSv8Zr7qJYCPki1FG8SML2JUu4CfgBMAa7MzEmZeT9wEfBbqo08zsvMBzq4NoCJZUOLi4Avl/bvUW3AcV9E/A64A5jVrt4HgQeAR6iea7u7HBoCXF/u8w7gi6X9lIh4uPT3q3Kf3crM54D9gUtLn78B3tvMtZIkSdKiqKW1tdlHe1SHiBhcdutbliqcHFRClBbAyDEjWsedtONCH3f82AndnyRJkqR+IyIml83nuuTmGn3vnPKlwYOongkzdEmSJEn9jMGrj2Xm3n1dgyRJkqR6+YyXJEmSJNXM4CVJkiRJNTN4SZIkSVLNDF6SJEmSVDODlyRJkiTVzOAlSZIkSTUzeEmSJElSzQxekiRJklQzg5ckSZIk1czgJUmSJEk1G9jXBUh1WGPIMMaPndDXZUiSJEmAM16SJEmSVDuDlyRJkiTVzOAlSZIkSTUzeEmSJElSzQxekiRJklQzg5ckSZIk1czgJUmSJEk1M3hJkiRJUs38AmX1S1NnPs3REw/v6zIWa34BtSRJUu9xxkuSJEmSambwkiRJkqSaGbwkSZIkqWYGL0mSJEmqmcFLkiRJkmpm8JIkSZKkmhm8JEmSJKlmBi9JkiRJqpnBS5IkSZJqZvCSJEmSpJoZvCRJkiSpZgYvSZIkSarZwL4uQG89EfE/wN7AXGAe8AwwJTO/3HDOKODSzHxfRAwGJgDbAa8BzwNHZua9C714SZIkaT4446WFKiI2B3YCNsrM9anC1MnAnu1O3Qv4cXl9HvACsE5mDgf2B1ZaKAVLkiRJvcAZLy1sqwHTMnM2QGZOA+6IiJciYrOGWayPAztGxNrAZsA+mTmvXPME8EQf1C5JkiTNF2e8tLDdDAyLiD9GxHcjYqvSfinVLBcR8X7g+cz8EzCcahni3L4pV5IkSVpwBi8tVJn5MrAxcBDwHHBZROwP/ATYIyKWoApgl/a074g4KCImRcSk2TNn92LVkiRJ0oJxqaEWujJ7NRGYGBEPA5/KzIsi4klgK+CjwObl9EeADSJiibalhl30ew5wDsDIMSNaaypfkiRJ6jFnvLRQRcS6EbFOQ9Mo4Kny+lLgdODxzJwKkJmPA5OAr0ZES+ljnYjYdSGWLUmSJC0QZ7y0sA0GzoyIFYA5wJ+plh0C/BT4DvDf7a75DNV28n+OiFmU7eQXTrmSJEnSgjN4aaHKzMnAFp0cew5YsoP2GcBnay5NkiRJqo1LDSVJkiSpZgYvSZIkSaqZwUuSJEmSambwkiRJkqSaGbwkSZIkqWYGL0mSJEmqmcFLkiRJkmpm8JIkSZKkmhm8JEmSJKlmBi9JkiRJqpnBS5IkSZJqZvCSJEmSpJoZvCRJkiSpZgP7ugCpDmsMGcb4sRP6ugxJkiQJcMZLkiRJkmpn8JIkSZKkmhm8JEmSJKlmBi9JkiRJqpnBS5IkSZJqZvCSJEmSpJoZvCRJkiSpZn6Pl/qlqTOf5uiJh/d1GU3x+8YkSZL6P2e8JEmSJKlmBi9JkiRJqpnBS5IkSZJqZvCSJEmSpJoZvCRJkiSpZgYvSZIkSaqZwUuSJEmSambwkiRJkqSaGbwkSZIkqWYGL0mSJEmqmcFLkiRJkmpm8JIkSZKkmhm8JEmSJKlmA/u6AC2+IqIVOC0zDy/vjwAGZ+aJXVyzBPBtYBugFXgN+DhwAvDrzPy/hnN3Aw7KzHER8fZy3SbAbOBJ4LDM/GMNtyZJkiT1Kme8tCBmAx+JiJV6cM2ewOrA+pk5EtgdeAm4FNir3bl7AZdGRAtwNTAxM9fOzPWAY4FVF/QGJEmSpIXB4KUFMQc4B/hi+wMRcVFE7NHw/uXycjXgmcycB5CZUzPzReAW4L0RsVo5f1lgO+AaYGvgjcz8flt/mTklM++s57YkSZKk3mXw0oI6G9gnIoY2ef7lwM4RMSUiJkTEhgCZORe4imrZIcAuwO2ZORMYAUzu5bolSZKkhcbgpQWSmTOAS4BDmzx/KrAu8GVgHnBrRGxbDjcuN9yrvG9aRBwUEZMiYtLsmbN7cqkkSZJUKzfXUG/4NnA/cGFD2xxKsC/PaC3VdiAzZwM3ATdFxD+A3YBbgbuB1SJiA2AL3gxhjwB70I3MPIdq6SMjx4xoXbBbkiRJknqPM15aYJn5AtUSwgMbmp8ENi6vdwWWBIiIjSJi9fJ6CWB94KnST2vp52Lgxsx8rVx/G7B0RHy2rfOI2CQitqrrniRJkqTeZPBSb5kANO5ueC6wVUT8FtgMeKW0rwJcFxG/Ax6imhk7q+G6S4ENgJ+0NZRAtjuwfUQ8HhGPACcCf6vnViRJkqTe1dLa6oos9T8jx4xoHXfSjn1dRlPGj53Q1yVIkiRpPkXE5Mwc3d15znhJkiRJUs0MXpIkSZJUM4OXJEmSJNXM4CVJkiRJNTN4SZIkSVLNDF6SJEmSVDODlyRJkiTVzOAlSZIkSTUzeEmSJElSzQxekiRJklQzg5ckSZIk1czgJUmSJEk1G9jXBUh1WGPIMMaPndDXZUiSJEmAM16SJEmSVDuDlyRJkiTVzOAlSZIkSTUzeEmSJElSzQxekiRJklQzg5ckSZIk1czgJUmSJEk1M3hJkiRJUs38AmX1S1NnPs3REw/v6zLUC/wibEmS1B844yVJkiRJNTN4SZIkSVLNDF6SJEmSVDODlyRJkiTVzOAlSZIkSTUzeEmSJElSzQxekiRJklQzg5ckSZIk1czgJUmSJEk1M3hJkiRJUs0MXpIkSZJUM4OXJEmSJNVsYF8XsCiIiLnAw8CSwBzgYuDbmTlvPvr6GvCrzLylk+MHA7My85Ie9rsjML68fTeQwKvAQ5m5X0/r7KD/5YEJwDbAa8BzwJHAA8C0zFxhQcco4xwCvJSZP4qI9YBLgXnAHsBFmfnB3hhHkiRJWpQYvCqvZuYogIhYBfgxMBQ4oacdZebx3Rz//vwUmJm/AH5RapwIHJGZk9qfFxEDM3POfAxxAfB74N2Z2RoR7wbWmZ9au5KZZze8/QhwRWZ+vbxvOnRFRAvQMj/hWJIkSVrYDF7tZOazEXEQcF9EnEi1HPNkYCywNHB2Zv4fQEQcBexLNWNzU2YeExEXAddn5hURcTKwC9Us2s2ZeUTp8+XMPDUiRgHfB5YFHgc+nZkvlmB1L7A1sAJwYGbe2VnNEfEZYDtgcKlx+4g4hirYDKIKN18r534KOARYCrgH+AJVwBoFfDwzW8vn8GfgzxExsGGc5YFrSk0DgWMz8/qIGAJcDqwODABOLPd/CvDhcv83ZebREXESMA14oow9NyK2Av6Thpm1juovYfAa4C5gM2Anqpk/SZIkaZFm8OpAZj4REUsAqwC7AtMzc5OIWBq4OyJuBt4L7AZslpmzImLFxj7K+92B95YZpI6W6l0C/Hdm3lGWKJ4AHFaODczMTSNiXGnfrpuyNwdGleA2DngHVThpAW6MiC2AGaWmLTJzTkScA+xFtbTwgSZmj14Fds3MmWVm8G7gemAc8GRmfqjc+9CIWLW0D+/o/jPzZxGxKVXY+na7gNdZ/c8C6wEHZObB3dQqSZIkLTIMXp1rKb93ANaPiD3K+6FUM0TbARdm5iyAzHyh3fUzqALNeRFxA1VA+aeIGAqskJl3lKaLgZ82nHJV+T0ZWKuJem/OzBcbav4Q1fNZUM2EvYdqpmoTYFJEACwDPA080kT/UH0m4yNiDNUs37CIWAl4CDi5zPBdl5l3R8Sscs65Hd1/Nzqr/1ng8cy8r6OLykzlQQDLrbRsD4aTJEmS6mXw6kBEvAuYS/Uf+i1Us1K/aHfOfwKtnfVRZpQ2BbalmlX6AtXGFc2aXX7Ppbm/0ysNr1uAkzLz/HY1fxG4IDO/0q59XWBURCzRzazXflTBc6Nyf1OBQZn5+4gYTTXDdUpEXJ+Z3yht21Pd/+epAlUzOqv/3e3u819k5jnAOQAjx4zo9G8jSZIkLWxuJ99ORKxM9dzVWeV5p18An4+IJcvx90TEcsDNwKcjYtnS3n6p4WBgaGbeSLV8cFTj8cycDrwYEW0bSuwL3EHv+AVwYKmTiFijzEzdAny8vCYi3hYR78jMP1Dt6nh82bSCiFg3InZu1+9Q4NkSurYHopwbVM+t/QA4DdioPPe1fGZeD3wR2LAX6pckSZIWS854VZaJiCm8uZ18W4AAOI9qqd/9JZQ8B+yWmT8vm2NMiojXgRuBYxv6HAJcGxGDqGZwvtjBuJ8Cvl/C2xPAAb1xM5l5Y0S8F/hNWVI4E9g7Mx+OiK8Ct5Rn2N4ADgb+WsY+jWpDjVfLfR7RrusfANdFxCTgfuBPpX0DqqWG84DXS59DgavKc3FLAF9a0Pp7+DFIkiRJi4yW1lZXZKn/GTlmROu4k3bs6zLUC8aPndDXJUiSJHUqIiZn5ujuznOpoSRJkiTVzOAlSZIkSTUzeEmSJElSzQxekiRJklQzg5ckSZIk1czgJUmSJEk1M3hJkiRJUs0MXpIkSZJUM4OXJEmSJNXM4CVJkiRJNTN4SZIkSVLNDF6SJEmSVDODlyRJkiTVbGBfFyDVYY0hwxg/dkJflyFJkiQBznhJkiRJUu0MXpIkSZJUM4OXJEmSJNXM4CVJkiRJNTN4SZIkSVLNDF6SJEmSVDODlyRJkiTVzO/xUr80debTHD3x8B5f53d/SZIkqQ7OeEmSJElSzQxekiRJklQzg5ckSZIk1czgJUmSJEk1M3hJkiRJUs0MXpIkSZJUM4OXJEmSJNXM4CVJkiRJNTN4SZIkSVLNDF6SJEmSVDODlyRJkiTVzOAlSZIkSTUzeEmSJElSzQb2dQELS0TMBR6muue/APtm5ku90O9awPWZOaIX+roI2AqYXpouyMwzFrTfTsYaC7yemfc0tO0HHAW0lJ8LMvPUUtf1mXlFL4y7OnBGZu5R3l8KDAcuBP4D+FVm3rKg40iSJEmLkrdM8AJezcxRABFxMXAI8L99W1KHjpyfgBMRAzJzbg8uGQu8DNxTrv8QcBiwQ2b+LSIGAfv2tI7uZObfgLbQ9XZgi8xcc376ioiBmTmnN+uTJEmS6vBWCl6Nfg2sDxARg4FrqWZblgSOy8xry0zWTcBdwBZAArtm5qsRsTFwATCrHKf0NQj4HjAamAN8KTNvj4j9gd2AAcAIYAKwFFWwmQ2My8wXOis2Ij4BHEs1C3VDZh5d2l8GTgN2BA6PiFfL+8HANGD/zHwmIg4FDi41PQocU97PjYhPAv8NfBk4ogQjMvM14NwOajke2BlYhiq0fS4zW9uPkZl7RcRWwHfKpa3AlsDbeHOG8GZglYiYUmo4sBy7onzGHd3LxDLuB4Cflc9SkiRJWqS95Z7xiogBwLZU/9EO8Bqwe2ZuBGwNTIiIlnJsHeDszBwOvAR8tLRfCByamZu36/4QgMwcCXwCuLiEMagC197AplQzbbMyc0OqELhfQx+nRMSU8jOyLM0bD2wDjAI2iYjdyrnLAb/LzM2Ae4EzgT0ysy0Yts3oHQNsmJnrAwdn5pPA94HTM3NUZt5Z6pvcxEd4VmZuUoLTMsBOHY1R2o4ADikzjR8EXm3X1y7A4w01ABARS3ZxLwArZOZWmWnokiRJ0mLhrTTjtUyZWVmLKmD8srS3AN+IiC2BeUAAq5Zjf8nMKeX1ZGCtiBhK9R/+d5T2HwAfKq/HUAUGMvOxiHgKeE85dntmzgRmRsR04LrS/jBl9q34l6WGEbErMDEznyvvf0Q1c3QNMBe4spy6LlV4+mVEQDW79kw59hDwo4i4ply3ILaOiKOAZYEVgUfKvXQ0xt3AaaXmqzJzaqmtO13dC8BlHV0UEQcBBwEst9KyPbwtSZIkqT7dBq+IWAJ4qDc2j+hjr2bmqBKcrqeanToD2AdYGdg4M9+IiCeBtlmq2Q3Xz6Wa4WmhWjbXkZZO2tv3Na/h/Ty6/jt01edrDc91tQCPdDALB/BhqrC2C/CViBjewTmPABsDt3U2WJm9+y4wOjOfjogTefOz+rcxMvPkiLgBGAf8JiK2o5ph7E5X9wLwSkeNmXkOcA7AyDEjOvsbSZIkSQtdt0sNM3Me8GBEvGMh1FO7zJwOHAocUZa0DQWeLaFra6DLjR7KTojTI2JMadqn4fCv2t5HxHuAdwB/WMCS7wW2ioiVyjLJTwB3dHDeH4CVI2LzMv6SETG8BOdhmXk71Y6FK1A9NzUTGNJw/TeBb5UNL4iIpctzW43aQta08mxc2yYZHY4REWtn5sOZOR6YBLy3yXvu8F6avFaSJEla5DT7jNdqwCMRcWtE/Kztp87C6pSZDwAPAnsBPwJGR8QkqtD0WBNdHACcHRG/5l+fW/ouMCAiHqZaDrd/Zs7uqIMe1PoM1cYXt5ea78/Mazs473WqIDQ+Ih4EplBtCjIA+GGp6QGq57peoloeuHt5luyDmXkjcDZwS0Q8QrW0cmC7MV6i2nDjYarlhPeVQ52NcVhE/K7U8yrVZiXN3HNn9yJJkiQtllpaW7tfkVV2p/s3Dc85SYuUkWNGtI47acceXzd+rPt1SJIkqXkRMTkzR3d3XlMzXiVgPQksWV7fB9y/QBVKkiRJ0ltEU8ErIj4LXAH8X1sTC747niRJkiS9JTT7jNchVF9YOwMgM/8ErFJXUZIkSZLUnzQbvGaXDQ8AiIiBdL6luiRJkiSpQbPB646IOJbqS4i3B37Km18ALEmSJEnqQrPB6xjgOaptxD8H3AgcV1dRkiRJktSfDOz+lH9+ifK55UeSJEmS1ANdBq+IuDwzP16+GPffnunKzPVrq0ySJEmS+onuZrwOK793qrsQSZIkSeqvugte1wMbASdl5r4LoR5JkiRJ6ne6C15LRcSngC0i4iPtD2bmVfWUJUmSJEn9R3fB62BgH2AFYOd2x1oBg5ckSZIkdaPL4JWZdwF3RcSkzDx/IdUkLbA1hgxj/NgJfV2GJEmSBHS/q+E2mXkb8KJLDSVJkiRp/nS31HAr4Db+fZkhuNRQkiRJkprS3VLDE8rvAxZOOZIkSZLU/3Q34wVARPw/4EJgJnAu1Rbzx2TmzTXWJkmSJEn9whJNnvfpzJwB7ACsAhwAnFxbVZIkSZLUjzQbvFrK73HAhZn5YEObJEmSJKkLzQavyRFxM1Xw+kVEDAHm1VeWJEmSJPUfzQavA4FjgE0ycxawJNVyQ0mSJElSN5raXAPYHJiSma8QMo0FAAAgAElEQVRExCepNtf4Tn1lSQtm6synOXri4X1dRr/kF1NLkiT1XLMzXt8DZkXEBsBRwFPAJbVVJUmSJEn9SLPBa05mtgK7At/JzO8AQ+orS5IkSZL6j2aXGs6MiC8DnwS2jIgBVM95SZIkSZK60eyM157AbODAzPw7EMAptVUlSZIkSf1IUzNeJWyd1vD+r/iMlyRJkiQ1pangFRHvB84E3gcsBQwAXs7MoTXWJkmSJEn9QrNLDc8CPgH8CVgG+Axwdl1FSZIkSVJ/0mzwIjP/DAzIzLmZeSEwtraqJEmSJKkfaXZXw1kRsRQwJSK+BTwDLFdfWZIkSZLUfzQ747Uv1XNdXwBeAYYBH62rKEmSJEnqT5rd1fCp8vJV4Kv1lSNJkiRJ/U+XwSsiHgZaOzuemev3ekWSJEmS1M90N+P1EWBV4Ol27WsCf6ulosVcROwOXAW8LzMf6+D4RcD1mXlFF31cBGwFTAcGAZdmZq/NNEbEbsAfM/PRhrYjqHarnAPMBSZk5iURMRE4IjMn9cK4o4H9MvPQiFgauAFYCfgmsD1wWmNNkiRJUn/RXfA6HTi2YakhABGxcjm2c12FLcY+AdwF7AWcuAD9HJmZV0TEIODRiLgkM//SGwUCuwHXA48CRMTBVMFn08ycERFDyzm9qoS3tgC3IbBkZo4q7y/rSV8RMSAz5/ZmfZIkSVJdugtea2XmQ+0bM3NSRKxVT0mLr4gYDHwA2Br4GXBiRLRQffn0NsBfgJaG84+nCq/LAPcAn8vM9ks7B5Xfr5RrtgVOpfrb3Qd8PjNnd9F+MrAL1UzWzVSzcbsAW0XEcVSbpBwLbJ2ZMwAyczpwcQf39z1gk1LvFZl5Qmn/lzEy84iI+BhwAtXs2fTM3DIixgJHAJ8GfgisHBFTSg3nU2bWImIHqmcJlwYeBw7IzJcj4kngAmAHqu+W+0nXfxFJkiRp0dDdroaDuji2TG8W0k/sBvw8M/8IvBARGwG7A+sCI4HPAls0nH9WZm6SmSOoPs+dGo6dUkLJVOAnmflsmf26CNgzM0dShazPd9G+Yhl/eHke76TMvIcqFB5ZZpueBYZk5uNN3N//ZOZoYH2q4LZ+R2OUc48HdszMDahC2T9l5rNUyxrvzMxRjWNHxErAccB2mbkR1QzZlxoufy0zx2SmoUuSJEmLje6C130R8dn2jRFxIDC5npIWa5/gzVmYn5T3W1I9ozU3M/8G3NZw/tYRcW/ZxGQbYHjDsbZg9HZg24jYgirA/aUEO6hmpbbson0G8BpwXkR8BJjVQc0tdLGBSjsfj4j7gQdKret1McbdwEXln58BTfYP8P7S790leH6K6pnCNp0uSYyIgyJiUkRMmj1zdg+GlCRJkurV3VLDw4CrI2If3gxao4GlqGY5VETE26jC04iIaKUKG63A1XQQbMos1XeB0Zn5dEScSAczjGWJ3URgDNVSwY60dNSYmXMiYlNgW6pnzr5Qamw8Z0ZEvBIR78rMJ7q4v3dSLRPcJDNfLBuADOpsjMw8OCI2Az5M9cXbozrru4N7+WVmfqKT4690dmFmngOcAzByzIhmw6QkSZJUuy5nvDLzH5m5BdXzNk+Wn69m5uaZ+ff6y1us7AFckplrZuZamTmM6pmuF4C9ImJARKxG9fwXvBmyppVnw/boqNOIGAhsRvWs02PAWhHx7nJ4X+COztpLv0Mz80aqEN0WfmYCQxqG+SZwdkQsX8ZcPiIOalfK8lShZ3pErAp8qJzb4RgRsXZm3puZxwPTqL50uxm/AT7Qdi8RsWxEvKfJayVJkqRFUrNfoHw7cHvNtSzuPgGc3K7tSuB9wJ+Ah4E/UgUlMvOliDi3tD9JtSFGo1PK5hdLAbcCV2Vma0QcAPy0BLL7gO+XTTT+rR1YEbi2zK61AF8sff8EODciDqUKfN8DBlMtLX0DeAOY0FhMZj4YEQ8AjwBPUC0lhCrAdTTGKRGxTmm7FXiQaov8LmXmcxGxP3Bp2XIeqme+/tj5VZIkSdKiraW11RVZ6n9GjhnROu6kHfu6jH5p/NgJ3Z8kSZL0FhERk8sGdF3qbnMNSZIkSdICMnhJkiRJUs0MXpIkSZJUM4OXJEmSJNXM4CVJkiRJNTN4SZIkSVLNDF6SJEmSVDODlyRJkiTVzOAlSZIkSTUzeEmSJElSzQxekiRJklQzg5ckSZIk1WxgXxcg1WGNIcMYP3ZCX5chSZIkAc54SZIkSVLtDF6SJEmSVDODlyRJkiTVzOAlSZIkSTUzeEmSJElSzQxekiRJklQzg5ckSZIk1czgJUmSJEk18wuU1S9Nnfk0R088vK/L6FV+IbQkSdLiyxkvSZIkSaqZwUuSJEmSambwkiRJkqSaGbwkSZIkqWYGL0mSJEmqmcFLkiRJkmpm8JIkSZKkmhm8JEmSJKlmBi9JkiRJqpnBS5IkSZJqZvCSJEmSpJoZvCRJkiSpZgYvSZIkSarZwL4uoKciYlXgdOD9wIvA68C3MvPq+ezvRODlzDw1Ir4G/Cozb5mPfkYBq2fmjeX9/sApQAJLAr8H9svMWfNTZxPj7QKsl5knz2d/SwJfBz4KzAZmASdk5k0R8SQwOjOn9ULd/6wzIlYGrgeWAg4FvgzsnZkvLeg4kiRJ0qJksQpeEdECXANcnJl7l7Y1gV3anTcwM+f0tP/MPH4ByhsFjAZubGi7LDO/UGr6MbAncOECjNHpeJn5M+BnC9Df14HVgBGZObsE3K0WuMp22tW5LfBYZn6qvL+zJ31FxIDMnNub9UmSJEl1WKyCF7AN8Hpmfr+tITOfAs4sM0wfBgYBy5WZlWuB/6CacTouM68FiIj/AfYDngaeAyaX9ouA6zPziojYGDgNGAxMA/bPzGciYiJwL7A1sAJwYHn/NWCZiBgDfLOx6IgYCCxHNUPXFhYvAFYu4x+QmX/tov1jwAnAXGA6sF0H4y1DNSv1hXIfM6iC2duBo8o9LQGcRRWo/kK11PQCqvD2WeCdmTm7fK7/AC5v/weIiGuAYeVz/k5mnhMRA4Dzy3itwAWZeXpEHAocDMwBHs3MvcrfaTRwHvCtcg9TgM2pZgVHZ+a0iPgk1SzYUuXz/a/MnBsRL5e/y47A4cBd7WuUJEmSFjWL2zNew4H7uzi+OfCpzNwGeA3YPTM3ogpJEyKipQSqvYANgY8Am7TvpCy7OxPYIzM3pgon/9twysDM3BQ4jGo53uvA8VQzXKMy87Jy3p4lVCSwInBdaT8LuCQz1wd+BJzRTfvxwI6ZuQGwSxfjNVoNGAPsBLQtP/wIsBYwEvhM+bwA3g38NTNndPip/qtPl89kNHBoRLyNavYtMnNEZo7kzVm9Y4ANy/0c3NhJZk5pdw+vth2LiPdRzQ5+IDNHUQXOfcrh5YDfZeZmmWnokiRJ0mJhcZvx+hcRcTZVuHgdOBv4ZWa+UA63AN+IiC2BeUAAqwIfBK5ue9YqIjpanrcuMAL4ZUQADACeaTh+Vfk9mSrIdOayMgPVUuo7kioEbU4VggB+QDXzQxftdwMXRcTlDWN355rMnAc8WpYNQvVZ/bS0/z0ibm+yr0aHRsTu5fUwYB3gD8C7IuJM4Abg5nL8IeBHZZbsmh6MsS2wMXBf+fyXAZ4tx+YCV3Z0UUQcBBwEsNxKy/ZgOEmSJKlei9uM1yPARm1vMvMQqv9IX7k0vdJw7j6lfeMya/IPquVxUC2H60oL8EiZiRmVmSMzc4eG47PL77k0EV4zs5VqtmvLTk7prJ7Wcv3BwHFUQWdKmWXqzuyG1y3tfrf3Z+AdETGkqw4jYizVMsfNy+zbA8CgzHwR2ACYCBxCtYwQqqWfZ1OFqMllyWUzWqie42v7/NfNzBPLsdc6e64rM8/JzNGZOXrpIUs3OZQkSZJUv8UteN0GDIqIzze0dTa1MRR4NjPfiIitgTVL+6+A3SNimRI0du7g2j8AK0fE5lAtPYyI4d3UNhPoKriMAR4vr++hWu4IVUC8q6v2iFg7M+8tm39Mowpg3Y3XkbuAj0bEEmUWbCxAmf07HzgjIpYqY65WnrNqNBR4MTNnRcR7qXaWJCJWApbIzCuBrwAblefJhmXm7cBRVM/DDW6yzluBPSJildL/iuX5N0mSJGmxtFgtNczM1ojYDTg9Io6i2oDiFeBoquVojX4EXBcRk4ApwGOlj/sj4rLS9hQd7KSXma9HxB5UQWQo1ef0baoZt87cDhxTnulq21xjz7L5xRLAVGD/0n4ocEFEHFnu4YBu2k+JiHWoZoJuBR4E/trBeN25kmqG8HfAH6k2rZhejh0HnES1NPE1qs+1/S6PPwcOjoiHqMLpb0p7ABeWsAXVtvADgB+Wz68FOD0zXypLB7uUmY9GxHHAzaXPN6hm0p5q8j4lSZKkRUpLa2t3q+7Un0TE4Mx8uSxX/C3VBhZ/7+u6etvIMSNax520Y1+X0avGj53Q1yVIkiSpnYiYnJmjuztvsZrxUq+4PiJWoNqm/ev9MXRJkiRJixqD11tMZo7t6xokSZKkt5rFbXMNSZIkSVrsGLwkSZIkqWYGL0mSJEmqmcFLkiRJkmpm8JIkSZKkmhm8JEmSJKlmBi9JkiRJqpnBS5IkSZJqZvCSJEmSpJoZvCRJkiSpZgP7ugCpDmsMGcb4sRP6ugxJkiQJcMZLkiRJkmpn8JIkSZKkmhm8JEmSJKlmBi9JkiRJqpnBS5IkSZJqZvCSJEmSpJoZvCRJkiSpZgYvSZIkSaqZX6CsfmnqzKc5euLhfV2GpH7EL2WXJC0IZ7wkSZIkqWYGL0mSJEmqmcFLkiRJkmpm8JIkSZKkmhm8JEmSJKlmBi9JkiRJqpnBS5IkSZJqZvCSJEmSpJoZvCRJkiSpZgYvSZIkSaqZwUuSJEmSambwkiRJkqSaDezrAtS7ImJV4HTg/cCLwOvAtzLz6hrHHA3sl5mHzuf1TwKTM/Oj5f0ewE6ZuX9E7A+cAiSwJPD7Mtas3qhdkiRJWhic8epHIqIFuAb4VWa+KzM3BvYC1qhz3MycNL+hq8HoiBjeybHLMnNUZg6nCpJ7LuBYkiRJ0kLljFf/sg3wemZ+v60hM58CzoyItYAfAMuVQ1/IzHsiYixwRGbuBBARZwGTMvOiiDgZ2AWYA9ycmUdExMeAE4C5wPTM3LKxj4jYFPg2sAzwKnBAZv6hzFztAiwLrA1cnZlHNdR+KnAssE9nNxcRA0v9L873JyRJkiT1AWe8+pfhwP2dHHsW2D4zN6KaMTqjq44iYkVgd2B4Zq4PnFQOHQ/smJkbUAWp9h4DtszMDcu532g4NqqMPRLYMyKGNRy7HNgoIt7dQZ97RsQUquWGKwLXdVW7JEmStKgxePVjEXF2RDwYEfdRPR91bkQ8DPwUWK+by2cArwHnRcRHgLZnqu4GLoqIzwIDOrhuKPDTiPgd1bNmjcsHb83M6Zn5GvAosGbDsblUz3J9uYM+L8vMUcDbgYeBIzu534MiYlJETJo9c3Y3tydJkiQtPAav/uURYKO2N5l5CLAtsDLwReAfwAbAaGCpctoc/vWfg0Hl2jnApsCVwG7Az0v7wcBxwDBgSkS8rV0NXwduz8wRwM5t/RWNaWgu/77U9QfAlsA7Orq5zGylmu3aspPj52Tm6MwcvfSQpTs6RZIkSeoTBq/+5TZgUER8vqFt2fJ7KPBMZs4D9uXN2aqngPUiYumIGEoV1IiIwcDQzLwROIxqmSARsXZm3puZxwPTqAJYo6FUSwIB9u9J8Zn5BtUs2WFdnDYGeLwn/UqSJEl9zc01+pHMbI2I3YDTI+Io4DngFeBoqme/riybY9xe2snMpyPicuAh4E/AA6W7IcC1ETEIaKGaMQM4JSLWKW23Ag8CWzWU8S3g4oj4ElUQ7KnzqWbUGu0ZEWOo/kfBVHoY6CRJkqS+1tLa2trXNUi9buSYEa3jTtqxr8uQ1I+MHzuhr0uQJC2CImJyZo7u7jyXGkqSJElSzQxekiRJklQzg5ckSZIk1czgJUmSJEk1M3hJkiRJUs0MXpIkSZJUM4OXJEmSJNXM4CVJkiRJNTN4SZIkSVLNDF6SJEmSVDODlyRJkiTVzOAlSZIkSTUb+P/bu/covao6zePfNNdGwt1R+RETcMVBSCBAcImDdsgwQLMQmsELFxHQBnE1Q+tgNwy0og1My4yIYdTuBgW8gM2lAUFU7KGhBVtoISABbzhchAca5B4wopCaP84pfXmtqgCpU29S+X7WqpX3PZe999l1Vup9au+za9ANkLqw2dRpnDrvtEE3Q5IkSQIc8ZIkSZKkzhm8JEmSJKljBi9JkiRJ6pjBS5IkSZI6ZvCSJEmSpI4ZvCRJkiSpYwYvSZIkSeqYwUuSJEmSOuYfUNakdP/i+zj22mPGpSz/ELMkSZKWlyNekiRJktQxg5ckSZIkdczgJUmSJEkdM3hJkiRJUscMXpIkSZLUMYOXJEmSJHXM4CVJkiRJHTN4SZIkSVLHDF6SJEmS1DGDlyRJkiR1zOAlSZIkSR0zeEmSJElSxwxekiRJktSxFT54VdXTI2w7sqreMwF131NVi9qvH1bVyVW1Vrtv06q6eBzq2LuqjnuJ53yjqjZY3rr7ypxRVQeOsH1BVaWqluteaftyk5dx3rhfqyRJkjTRVh90A16OJH/XZflVNQWY0r7dJckjVbUucGb7dUiSB4C3L2c9qye5HLj8pZyXZM/lqXcUM4ADgfOHN7Rha1/gPuCtwLUd1Dumjq5VkiRJmlArZfCqqo8BTyf5ZFVdC9wI7AJsALwvyXVVtRrwCWAesBbw2SR/3waorwEbAmsAf5Xka1U1A/gmcA2wE/AnvXUmebqqjgTuq6qNgPWAryeZVVVbA+cAa9KMIu6X5M52VO7DwBBwW5KDq+pc4DFgO2BhVS0C5iY5qt23BNgSmA4cBhzStufGJIe2138PMBdYt23z9cCbgQD7JFlSVYcDR7Rt+hlwcJJftnU81Z7/auAvk1zc9tUbqupW4ItJTm/79HbgAuAA2uDV9v9rgS3afz+d5Ix232XANGBtYEGSM/u+dycBjyRZ0L4/BXgIuKitZz2a+/ID7fdx+FqXABcCmwGrAScluQBJkiRpJbDCTzV8kVZP8kbgg8CJ7bb3AU8m2RHYETi8qjYHfgXsm2R7mmBxWjvCBfAfgS8l2S7Jvf2VJHkKuBuY2bfrSJqQMYcmJNzfhrETgPlJtgX+vOf41wO7JjlmhGvZEJgPfAi4Ajgd2BqYXVVzRjh+Jk2o3Bp4Ativ3X5Jkh3bun/U9sew1wA7A3vRBC6A44DrksxpQxc0YeurwKXAXlW1Rk8ZWwK7A28ETuzZ994kO7T9cHRVbdzX3i/QhMnhEbX9gfNoRtuuavtwW+DWvvP2AB5Ism2SWcC3RugLSZIkaYU0WYLXJe2/N9NMmQPYDXhPO4JzI7AxTUiZAvzPqroN+L9AAa9qz7k3yQ3LqGvKCNu+BxxfVccC05MsoQlPFyd5BCDJYz3HX5Tk+VHKvyLJELAIeCjJoiRLgTt6rq3X3UmGQ0rv9c+qquvaEbWDaMLbsMuSLE3yQ3537S9QVWsCe7bHPkXTh7v1HHJlkmfb63u4p5yjq+oHwA00I18vCKlJ7gEerart2vJuSfIo8H3gsHY0bXaSxX1NWgTsWlWnVtVbkjw5QpuPqKqbquqmZxc/O9JlSZIkSQMxWYLX8Kfs5/nd9MkpwH9rR3DmJNk8ybdpQsgrgR3a0ZWHaKbFATwzViVVNZUm2Py0d3uS84G9aabDXVVV89v6h0Ypaqx6hq9lac/r4fcjTQ3tPab3+s8FjkoyG/g4v7vG/nNGCpLQjDCtDyxqp/vtTDMCNmq9VTUP2BXYqR1pu6Wv3mGfBw6lmUp5NkCS79A8Rxbgy/2LpyT5KbADTQD7m6r6aH+hSc5MMjfJ3LWmrjXKZUmSJEkTb7IEr5FcBXxgeApcVb2+ql5BEyYeTvKbqtqF5lmqZWqfDfsczQjQ4337tgDuap9zuhzYBrgaeOfwVLv2ubCJNBV4sL3+g17E8Yvbc4YdAPxpkhlJZgCbA7tV1TpjlLE+8Hj7LNmWwJtGOe5SmmC3I833iaqaTvN9OYtmOuL2vSdU1abAL5N8Bfhk/35JkiRpRbYyLK6xTlXd3/P+Uy/yvM/TjE4tbJ/h+gXNghnnAVdU1U00zxH9eBnlXNOe/wc0geGkEY55F/DuqvoN8O/AXyd5rF044l+q6nma0Z9DX2Tbx8NHaKYH3kszSjR17MO5DXiunSZ4Ic3zW+8f3pnkmaq6HnjbGGV8Cziyncb5E5rphr8nya+r6hrgiZ4pl/OAv2j78Gmg/88FzAb+d1UtBX4DfGAZ1yNJkiStMKYMDY02G07qRruoxkLgHUnu7KKO2TvPGtrz5N3HpaxT5502LuVIkiRp8qmqm5PMXdZxk3mqoVZAVbUVzfL2V3cVuiRJkqQVzcow1VCTSLuS4haDbockSZI0kRzxkiRJkqSOGbwkSZIkqWMGL0mSJEnqmMFLkiRJkjpm8JIkSZKkjhm8JEmSJKljBi9JkiRJ6pjBS5IkSZI6ZvCSJEmSpI4ZvCRJkiSpY6sPugFSFzabOo1T55026GZIkiRJgCNekiRJktQ5g5ckSZIkdczgJUmSJEkdM3hJkiRJUscMXpIkSZLUMYOXJEmSJHXM4CVJkiRJHTN4SZIkSVLH/APKmpTuX3wfx157zKCbIUmSpI6dOu+0QTfhRXHES5IkSZI6ZvCSJEmSpI4ZvCRJkiSpYwYvSZIkSeqYwUuSJEmSOmbwkiRJkqSOGbwkSZIkqWMGL0mSJEnqmMFLkiRJkjpm8JIkSZKkjhm8JEmSJKljBi9JkiRJ6linwauqNquqr1XVnVV1V1V9pqrWGody51XV11/iOTOq6sCe93Or6oxlnHNPVS1qv35YVScPt7+qNq2qi1/eFbygjr2r6riXeM43qmqD5a27r8wX9E/P9gVVlaparnul7ctNXsZ5436tkiRJ0kTrLHhV1RTgEuCyJDOBmcAfAv+rwzpXH2P3DOC3wSLJTUmOfhHF7pJkNvBGYAvgzPb8B5K8fTmaS1WtnuTyJJ94Kecl2TPJE8tT9whm0NM/AG3Y2he4D3jrONf3onR0rZIkSdKEGiuoLK/5wK+SnAOQ5Pmq+hBwb1XdCWyZ5CiAdvTqk0muraq/BXakCWkXJzmxPWYP4NPAI8DC4Uqq6mPApjTB4ZGqOh74MvCK9pCjkvwr8AngDVV1K/BF4Bbgw0n2qqp1gf8DzAWGgI8n+cfei0nydFUdCdxXVRsB6wFfTzKrqrYGzgHWpAmz+yW5s6reA3y4LfO2JAdX1bnAY8B2wMKqWgTMTXJUu28JsCUwHTgMOATYCbgxyaHtNd/TtnVd4JvA9cCbgQD7JFlSVYcDR7Rt+hlwcJJftnU81Z7/auAvk1zc3z9JTgd2AW4HLgAOAK7t6fPX0gTR1wKfTnJGu+8yYBqwNrAgyZm9/VhVJwGPJFnQvj8FeAi4qK1nPZr78gNJruu51iXAhcBmwGrASUkuQJIkSVoJdDnVcGvg5t4NSZ4C7mHswHdCkrnANsAfVdU2VbU2cBbwNuAtNIGh1w40geNA4GHgvyTZHngXMDyd8DjguiRz2lDR6yPAk0lmJ9kG+OeRGta2/26a0bteR9KEjDk0IeH+NoydAMxPsi3w5z3Hvx7YNckxI1SzIU1o/RBwBXA6TV/Orqo5Ixw/E/hskq2BJ4D92u2XJNmxrftHwPt6znkNsDOwF03ggpH75wDgq8ClwF5VtUZPGVsCu9OMBJ7Ys++9SXZo++Hoqtq4r71foAmTwyNq+wPn0Yy2XdX24bbArX3n7QE8kGTbJLOAb43QF5IkSdIKqcvgNYVmpGek7WN5Z1UtpBmR2hrYiuZD/t1J7kwyBHyl75zLkyxpX68BnNWOJF3Unr8suwKfHX6T5PExjh2p/d8Djq+qY4HpbVvm04zYPdKW+VjP8RcleX6U8q9or3ER8FCSRUmWAnfQjOr1uzvJcEi5ueeYWVV1XdsPB9H05bDLkixN8kPgVSM1oqrWBPZsj30KuBHYreeQK5M8217fwz3lHF1VPwBuoBn5ekFITXIP8GhVbdeWd0uSR4HvA4e1o2mzkyzua9IiYNeqOrWq3pLkyRHafERV3VRVNz27+NmRLkuSJEkaiC6D1x00ox6/VVXr0XxAf7Sv7rXb/ZvTTM37z+3I05XD+xg5xA17puf1h2imrm3b1r/mi2jraCHxBapqKk2w+Wnv9iTnA3vTTIe7qqrmL6PMZ0bZDjCcGJb2vB5+P9JIYe8xz/cccy7NNMvZwMf5XT/2nzNaEN4DWB9Y1E7325lmBGzUeqtqHk2I3akdabulr95hnwcOpZlKeTZAku/QPEcW4MvtNM3fSvJTmpHNRcDfVNVH+wtNcmaSuUnmrjV1uddwkSRJksZNl8HramCd4Q/QVbUacBrwGZrpenOq6g+qahrNdDVonu95Bniyql4F/HG7/cfA5lX1uvZ9bwDotz7wYDtKdDDN80AAi4Gpo5zzbeCo4TdVtWH/Ae1zYJ+jGQF6vG/fFsBd7XNOl9NMk7yaZvRu4/aYjcZocxemAg+2UwAPehHH9/fPAcCfJpmRZAawObBbVa0zRhnrA4+3z5JtCbxplOMupQl2OwJXAVTVdODhJGfRTEfcvveEqtoU+GWSrwCf7N8vSZIkrcg6C17tdLl9gbe3i2k8CixNcgrwXZrwtYjmQ/TC9pwf0IyS3EEzEvLddvuvaBaKuLKqrgfuHaPqzwGHVNUNNM9SDY8u3QY8V1U/aBf56HUysGFV3d5Ok9ulZ981VXU78G/Az4H3j1Dnu4Db24UptgS+lOQO4BTgX9oyPzVGm7vwEZrpgf9EE1yXpbd/TqB5fuvK4Z1JnqFZxONtY5TxLZqRr9uAk2imG/6eJL8GrgEu7JlyOQ+4tapuoRCge5gAAAdiSURBVHlObUHfabOBf2v7+ASa75kkSZK0UpgyNLTMGXbjoqreTLNQw39NcvOyjtfk1S6qsRB4R5I7u6hj9s6zhvY8efcuipYkSdIK5NR5pw20/qq6uV0ccExdLif/Au2S7tMnqj6tmKpqK+DrwKVdhS5JkiRpRTNhwUsCaFdS3GLQ7ZAkSZImUpeLa0iSJEmSMHhJkiRJUucMXpIkSZLUMYOXJEmSJHXM4CVJkiRJHTN4SZIkSVLHDF6SJEmS1DGDlyRJkiR1zOAlSZIkSR0zeEmSJElSx1YfdAOkLmw2dRqnzjtt0M2QJEmSAEe8JEmSJKlzBi9JkiRJ6pjBS5IkSZI6ZvCSJEmSpI5NGRoaGnQbpHFXVYuBnwy6HaugTYBHBt2IVZD9Phj2+8SzzwfDfh8M+30wXk6/T0/yymUd5KqGmqx+kmTuoBuxqqmqm+z3iWe/D4b9PvHs88Gw3wfDfh+MLvvdqYaSJEmS1DGDlyRJkiR1zOClyerMQTdgFWW/D4b9Phj2+8SzzwfDfh8M+30wOut3F9eQJEmSpI454iVJkiRJHXNVQ006VbUHsABYDfh8kk8MuEmrhKq6B1gMPA8850pM3aiqs4G9gIeTzGq3bQRcAMwA7gHemeTxQbVxshmlzz8GHA78oj3s+CTfGEwLJ6eqmgZ8CXg1sBQ4M8kC7/dujdHvH8N7vhNVtTbwHWAtms/mFyc5sao2B/4B2AhYCByc5NeDa+nkMka/nwv8EfBke+ihSW4djzod8dKkUlWrAZ8F/hjYCjigqrYabKtWKbskmWPo6tS5wB59244Drk4yE7i6fa/xcy6/3+cAp7f3+xw/gHbiOeCYJG8A3gT8Wfv/ufd7t0brd/Ce78qzwPwk2wJzgD2q6k3AqTR9PhN4HHjfANs4GY3W7wB/0XOvj0voAoOXJp83Aj9Lclf7W6F/APYZcJukcZPkO8BjfZv3Ab7Yvv4i8CcT2qhJbpQ+V8eSPJhkYft6MfAjoPB+79QY/a6OJBlK8nT7do32awiYD1zcbvdeH2dj9HtnDF6abAq4r+f9/fgDY6IMAd+uqpur6ohBN2YV86okD0LzoQn4DwNuz6riqKq6rarOrqoNB92YyayqZgDbATfi/T5h+vodvOc7U1WrVdWtwMPAPwH/D3giyXPtIX6e6UB/vycZvtdPae/106tqrfGqz+ClyWbKCNtcunNi/Kck29NM8/yzqnrroBskdehvgdfRTE95EDhtsM2ZvKpqXeAfgQ8meWrQ7VlVjNDv3vMdSvJ8kjnAZjSzd94wwmF+nhln/f1eVbOA/wFsCexI83zdseNVn8FLk839wLSe95sBDwyoLauUJA+0/z4MXErzg0MT46Gqeg1A++/DA27PpJfkofYH9lLgLLzfO1FVa9B8+D8vySXtZu/3jo3U797zEyPJE8C1NM/XbVBVwwvh+XmmQz39vkc73XYoybPAOYzjvW7w0mTzfWBmVW1eVWsC+wOXD7hNk15VvaKqpg6/BnYDbh9sq1YplwOHtK8PAb42wLasEoY/+Lf2xft93FXVFOALwI+SfKpnl/d7h0brd+/57lTVK6tqg/b1HwK70jxbdw3w9vYw7/VxNkq//7jnFztTaJ6rG7d73T+grEmnqvYEPk2znPzZSU4ZcJMmvaragmaUC5olWc+337tRVV8F5gGbAA8BJwKXARcCrwV+DrwjiYtBjJNR+nwezZSrIZolzd8//NyRxkdV7QxcByyiWdYc4Hia54283zsyRr8fgPd8J6pqG5rFM1ajGRS5MMlftz9bh5eTvwV4dzsKo3EwRr//M/BKmsdXbgWO7FmEY7kYvCRJkiSpY041lCRJkqSOGbwkSZIkqWMGL0mSJEnqmMFLkiRJkjpm8JIkSZKkjhm8JElaSVXVuCxx3HWZkiSDlyRJkiR1bvVBN0CSJI2fqpoOnE3zB0B/ARyW5OdV9TrgPJo/FvpN4L8nWXc5y3wHzR+Ufh54Mslbq2pr4BxgTZpf8O6X5M5xvUhJWgk54iVJ0uTyGeBLSbahCVpntNsXAAuS7Ag8ME5lfhTYPcm2wN7ttiPbeuYAc4H7X/aVSNIkYvCSJGly2Qk4v339ZWDnnu0Xta/P7z/pZZb5XeDcqjqcZiQN4HvA8VV1LDA9yZKXWJckTUoGL0mSJrehrspMciTwV8A04Naq2jjJ+TSjX0uAq6pqfgf1S9JKx+AlSdLk8q/A/u3rg4Dr29c3APu1r/fvP+nllFlVr0tyY5KPAo8A06pqC+CuJGcAlwPbvKyrkKRJZsrQUBe/CJMkSV2rqqW88HmtTwGX0CyEsQkvXAhjJvAVYApwJXBEklrOMi8BZrZlXg18EDgOeDfwG+DfgQOTPDZuFy1JKymDlyRJq4CqWgdYkmSoqvYHDkiyz6DbJUmrCpeTlyRp1bAD8JmqmgI8Abx3wO2RpFWKI16SJEmS1DEX15AkSZKkjhm8JEmSJKljBi9JkiRJ6pjBS5IkSZI6ZvCSJEmSpI4ZvCRJkiSpY/8fHiC6EEm/2VcAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1c4b896780>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(12,8))\n",
    "sns.set_color_codes(\"muted\")\n",
    "sns.barplot(x='Log Loss', y='Classifier', data=log, color=\"g\")\n",
    "plt.xlabel('Log Loss')\n",
    "plt.title('Classifier Log Loss')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 138,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Applications/anaconda3/envs/Python36/lib/python3.6/site-packages/sklearn/discriminant_analysis.py:510: RuntimeWarning: overflow encountered in exp\n",
      "  np.exp(prob, prob)\n"
     ]
    }
   ],
   "source": [
    "# Predict Test Set\n",
    "favorite_clf = LinearDiscriminantAnalysis()\n",
    "favorite_clf.fit(X_train, y_train)\n",
    "test_predictions = favorite_clf.predict_proba(test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 139,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(792, 192)"
      ]
     },
     "execution_count": 139,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 140,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(198, 192)"
      ]
     },
     "execution_count": 140,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_test.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 141,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "pred = favorite_clf.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 142,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "             precision    recall  f1-score   support\n",
      "\n",
      "          0       1.00      1.00      1.00         2\n",
      "          1       1.00      1.00      1.00         2\n",
      "          2       1.00      1.00      1.00         2\n",
      "          3       1.00      1.00      1.00         2\n",
      "          4       1.00      1.00      1.00         2\n",
      "          5       1.00      1.00      1.00         2\n",
      "          6       1.00      1.00      1.00         2\n",
      "          7       0.67      1.00      0.80         2\n",
      "          8       1.00      1.00      1.00         2\n",
      "          9       1.00      1.00      1.00         2\n",
      "         10       1.00      1.00      1.00         2\n",
      "         11       1.00      1.00      1.00         2\n",
      "         12       1.00      1.00      1.00         2\n",
      "         13       1.00      1.00      1.00         2\n",
      "         14       1.00      1.00      1.00         2\n",
      "         15       1.00      1.00      1.00         2\n",
      "         16       1.00      1.00      1.00         2\n",
      "         17       1.00      0.50      0.67         2\n",
      "         18       1.00      1.00      1.00         2\n",
      "         19       1.00      1.00      1.00         2\n",
      "         20       1.00      1.00      1.00         2\n",
      "         21       1.00      1.00      1.00         2\n",
      "         22       1.00      1.00      1.00         2\n",
      "         23       0.67      1.00      0.80         2\n",
      "         24       1.00      0.50      0.67         2\n",
      "         25       1.00      1.00      1.00         2\n",
      "         26       1.00      1.00      1.00         2\n",
      "         27       1.00      1.00      1.00         2\n",
      "         28       1.00      1.00      1.00         2\n",
      "         29       1.00      1.00      1.00         2\n",
      "         30       1.00      1.00      1.00         2\n",
      "         31       0.67      1.00      0.80         2\n",
      "         32       1.00      1.00      1.00         2\n",
      "         33       1.00      1.00      1.00         2\n",
      "         34       1.00      1.00      1.00         2\n",
      "         35       1.00      1.00      1.00         2\n",
      "         36       1.00      1.00      1.00         2\n",
      "         37       1.00      1.00      1.00         2\n",
      "         38       1.00      1.00      1.00         2\n",
      "         39       1.00      1.00      1.00         2\n",
      "         40       1.00      0.50      0.67         2\n",
      "         41       1.00      1.00      1.00         2\n",
      "         42       1.00      1.00      1.00         2\n",
      "         43       1.00      1.00      1.00         2\n",
      "         44       1.00      1.00      1.00         2\n",
      "         45       1.00      1.00      1.00         2\n",
      "         46       1.00      1.00      1.00         2\n",
      "         47       1.00      1.00      1.00         2\n",
      "         48       1.00      1.00      1.00         2\n",
      "         49       1.00      1.00      1.00         2\n",
      "         50       1.00      1.00      1.00         2\n",
      "         51       1.00      1.00      1.00         2\n",
      "         52       1.00      1.00      1.00         2\n",
      "         53       1.00      1.00      1.00         2\n",
      "         54       1.00      1.00      1.00         2\n",
      "         55       1.00      1.00      1.00         2\n",
      "         56       1.00      1.00      1.00         2\n",
      "         57       1.00      1.00      1.00         2\n",
      "         58       1.00      1.00      1.00         2\n",
      "         59       1.00      1.00      1.00         2\n",
      "         60       1.00      1.00      1.00         2\n",
      "         61       1.00      1.00      1.00         2\n",
      "         62       1.00      1.00      1.00         2\n",
      "         63       1.00      1.00      1.00         2\n",
      "         64       1.00      1.00      1.00         2\n",
      "         65       1.00      1.00      1.00         2\n",
      "         66       1.00      1.00      1.00         2\n",
      "         67       1.00      1.00      1.00         2\n",
      "         68       1.00      1.00      1.00         2\n",
      "         69       1.00      1.00      1.00         2\n",
      "         70       1.00      1.00      1.00         2\n",
      "         71       1.00      1.00      1.00         2\n",
      "         72       1.00      1.00      1.00         2\n",
      "         73       1.00      1.00      1.00         2\n",
      "         74       1.00      1.00      1.00         2\n",
      "         75       1.00      1.00      1.00         2\n",
      "         76       1.00      1.00      1.00         2\n",
      "         77       1.00      1.00      1.00         2\n",
      "         78       1.00      1.00      1.00         2\n",
      "         79       1.00      1.00      1.00         2\n",
      "         80       1.00      1.00      1.00         2\n",
      "         81       1.00      1.00      1.00         2\n",
      "         82       1.00      1.00      1.00         2\n",
      "         83       1.00      1.00      1.00         2\n",
      "         84       1.00      1.00      1.00         2\n",
      "         85       1.00      1.00      1.00         2\n",
      "         86       1.00      1.00      1.00         2\n",
      "         87       1.00      1.00      1.00         2\n",
      "         88       1.00      1.00      1.00         2\n",
      "         89       1.00      1.00      1.00         2\n",
      "         90       1.00      1.00      1.00         2\n",
      "         91       1.00      1.00      1.00         2\n",
      "         92       1.00      1.00      1.00         2\n",
      "         93       1.00      1.00      1.00         2\n",
      "         94       1.00      1.00      1.00         2\n",
      "         95       1.00      1.00      1.00         2\n",
      "         96       1.00      1.00      1.00         2\n",
      "         97       1.00      1.00      1.00         2\n",
      "         98       1.00      1.00      1.00         2\n",
      "\n",
      "avg / total       0.99      0.98      0.98       198\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(classification_report(y_test,pred))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 143,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[2 0 0 ..., 0 0 0]\n",
      " [0 2 0 ..., 0 0 0]\n",
      " [0 0 2 ..., 0 0 0]\n",
      " ..., \n",
      " [0 0 0 ..., 2 0 0]\n",
      " [0 0 0 ..., 0 2 0]\n",
      " [0 0 0 ..., 0 0 2]]\n"
     ]
    }
   ],
   "source": [
    "print(confusion_matrix(y_test,pred))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# putting everything in grid search mode\n",
    "parameters =  \"insert params dict here\"\n",
    "\n",
    "grid_obj = GridSearchCV(clf, parameters, scoring='log_loss')\n",
    "grid_obj = grid_obj.fit(X_train, y_train)\n",
    "\n",
    "clf = grid_obj.best_estimator_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 144,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# imports\n",
    "import numpy as np                     # numeric python lib\n",
    "\n",
    "import matplotlib.image as mpimg       # reading images to numpy arrays\n",
    "import matplotlib.pyplot as plt        # to plot any graph\n",
    "import matplotlib.patches as mpatches  # to draw a circle at the mean contour\n",
    "\n",
    "from skimage import measure            # to find shape contour"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 145,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import scipy.ndimage as ndi            # to determine shape centrality"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 146,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# matplotlib setup\n",
    "%matplotlib inline\n",
    "from pylab import rcParams\n",
    "rcParams['figure.figsize'] = (6, 6)      # setting default size of plots"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 147,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import os"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 148,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'/Users/pradmishra/Downloads/images'"
      ]
     },
     "execution_count": 148,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "os.getcwd()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 160,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "os.chdir('/Users/pradmishra')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 161,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'/Users/pradmishra'"
      ]
     },
     "execution_count": 161,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "os.getcwd()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 162,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "img = mpimg.imread('figure.jpg')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 163,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[[255, 255, 255, 255],\n",
       "        [255, 255, 255, 255],\n",
       "        [255, 255, 255, 255],\n",
       "        ..., \n",
       "        [255, 255, 255, 255],\n",
       "        [255, 255, 255, 255],\n",
       "        [255, 255, 255, 255]],\n",
       "\n",
       "       [[255, 255, 255, 255],\n",
       "        [255, 255, 255, 255],\n",
       "        [255, 255, 255, 255],\n",
       "        ..., \n",
       "        [255, 255, 255, 255],\n",
       "        [255, 255, 255, 255],\n",
       "        [255, 255, 255, 255]],\n",
       "\n",
       "       [[255, 255, 255, 255],\n",
       "        [255, 255, 255, 255],\n",
       "        [255, 255, 255, 255],\n",
       "        ..., \n",
       "        [255, 255, 255, 255],\n",
       "        [255, 255, 255, 255],\n",
       "        [255, 255, 255, 255]],\n",
       "\n",
       "       ..., \n",
       "       [[255, 255, 255, 255],\n",
       "        [255, 255, 255, 255],\n",
       "        [255, 255, 255, 255],\n",
       "        ..., \n",
       "        [255, 255, 255, 255],\n",
       "        [255, 255, 255, 255],\n",
       "        [255, 255, 255, 255]],\n",
       "\n",
       "       [[255, 255, 255, 255],\n",
       "        [255, 255, 255, 255],\n",
       "        [255, 255, 255, 255],\n",
       "        ..., \n",
       "        [255, 255, 255, 255],\n",
       "        [255, 255, 255, 255],\n",
       "        [255, 255, 255, 255]],\n",
       "\n",
       "       [[255, 255, 255, 255],\n",
       "        [255, 255, 255, 255],\n",
       "        [255, 255, 255, 255],\n",
       "        ..., \n",
       "        [255, 255, 255, 255],\n",
       "        [255, 255, 255, 255],\n",
       "        [255, 255, 255, 255]]], dtype=uint8)"
      ]
     },
     "execution_count": 163,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "img"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 170,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# using image processing module of scipy to find the center of the leaf\n",
    "cx, cy, cz = ndi.center_of_mass(img)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 171,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[[255, 255, 255, 255],\n",
       "        [255, 255, 255, 255],\n",
       "        [255, 255, 255, 255],\n",
       "        ..., \n",
       "        [255, 255, 255, 255],\n",
       "        [255, 255, 255, 255],\n",
       "        [255, 255, 255, 255]],\n",
       "\n",
       "       [[255, 255, 255, 255],\n",
       "        [255, 255, 255, 255],\n",
       "        [255, 255, 255, 255],\n",
       "        ..., \n",
       "        [255, 255, 255, 255],\n",
       "        [255, 255, 255, 255],\n",
       "        [255, 255, 255, 255]],\n",
       "\n",
       "       [[255, 255, 255, 255],\n",
       "        [255, 255, 255, 255],\n",
       "        [255, 255, 255, 255],\n",
       "        ..., \n",
       "        [255, 255, 255, 255],\n",
       "        [255, 255, 255, 255],\n",
       "        [255, 255, 255, 255]],\n",
       "\n",
       "       ..., \n",
       "       [[255, 255, 255, 255],\n",
       "        [255, 255, 255, 255],\n",
       "        [255, 255, 255, 255],\n",
       "        ..., \n",
       "        [255, 255, 255, 255],\n",
       "        [255, 255, 255, 255],\n",
       "        [255, 255, 255, 255]],\n",
       "\n",
       "       [[255, 255, 255, 255],\n",
       "        [255, 255, 255, 255],\n",
       "        [255, 255, 255, 255],\n",
       "        ..., \n",
       "        [255, 255, 255, 255],\n",
       "        [255, 255, 255, 255],\n",
       "        [255, 255, 255, 255]],\n",
       "\n",
       "       [[255, 255, 255, 255],\n",
       "        [255, 255, 255, 255],\n",
       "        [255, 255, 255, 255],\n",
       "        ..., \n",
       "        [255, 255, 255, 255],\n",
       "        [255, 255, 255, 255],\n",
       "        [255, 255, 255, 255]]], dtype=uint8)"
      ]
     },
     "execution_count": 171,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "img"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 166,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x111239828>"
      ]
     },
     "execution_count": 166,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAECCAYAAADw0Rw8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJzt3Xl8VPW5+PHPZLKRjSwkIXxZAiRCSIAIIbihgKKAC9prVW5ttXJL2592cfn9tLZVvHVBr9ra5fZqWy1dRLFVsYr7cl0RA0S2gAEJhC+B7AlkIdv8/piZGLIOyTlzTmae9+uVVzJnzsw8OUmefOc53/N8HS6XCyGEEIErxOoAhBBCmEsSvRBCBDhJ9EIIEeAk0QshRICTRC+EEAFOEr0QQgS4ULOeWCm1GHgMcAJ/1FqvNuu1hBBC9M1hxjx6pZQT+AJYBBwCPgOWa613Gf5iQggh+mVW6SYf2Ku1/lJr3QI8Aywz6bWEEEL0w6xEr4DSLrcPebYJIYTwM7Nq9I5etp1UI1JKrQRWAkRFRc3OyMgwKRQhhAhM27Ztq9RaJw+0n1mJ/hAwrsvtscDhrjtorZ8AngBYsmSJ69VXXzUpFCGECExKqQO+7GdWov8MyFRKTQQ0cA3w7ya9lhBCiH6YUqPXWrcBNwGvA0XAOq31TjNeSwghRP9Mm0evtd4AbDDr+YUQQvhGrowVQogAJ4leCCECnCR6IYQIcJLohRAiwEmiF0KIACeJXgghApwkeiGECHCS6IUQIsBJohdCiAAniV4IIQKcJHohhAhwkuiFECLASaIXQogAJ4leCCECnCR6IYQIcJLohRAiwEmiF0KIACeJXgghAlzQJfrm5mby8/PJzs7m7rvvBuD6668nNzeX3NxcCgsLLY5QCCGMZdqasXYVERHBO++8Q0REBOeccw5LliwBkAQvhAhYQTeidzgcjBgxgoSEBObPn88ZZ5wBwIwZM7j55ps5ceKExREKIYSxgm5ED+B0Ojl06BBXXHEFO3bs4IEHHiAxMZGVK1fy4IMPctddd520f15eHgC7d+9m6tSpVoQshLCZgoICq0PwWVAmeoD4+Hjmz5/Pa6+9xm233QbAt7/9bR5++OEe+3p/oHl5ecPqhyuEEBCEpZuKigpqa2tpamrirbfeYurUqZSVleFyuXjxxRfJycmxOkQhhDBU0I3ok5OTO7/+4IMPTrrvV7/6lb/DEUII0wXdiF4IIYKNJHohhAhwkuiFsKkvK45bHYIIEJLohbCh2sYWljz2wcA7CuEDSfRC2NAbO49yoq3D6jBEgBjSrBulVAlwDGgH2rTWeUqpROBZIB0oAa7SWtcMLUwhgsvL28usDkEEECNG9Au01rla6zzP7TuAt7XWmcDbnttCCB+t+6yU97+oICstzupQRIAwo3SzDFjj+XoNcLkJryFEwHp5exnjEkewcGoy7R0uq8MRAWCoid4FvKGU2qyUWunZlqq1LgPwfE4Z4msIETRqG1v4eG8lS6enkRIbSU1ji9UhiQAw1Ctjz9ZaH1ZKpQBvKqV2+/pAzz+GlZ6vhxiGEIHhh88UkhYfyR2Lp7L5QA27Dtdz7mnJAz9QiH4MaUSvtT7s+VwOvADkA0eVUmkAns/lfTz2Ca11ntY6LykpaShhCBEwvKN5h8NBcmwEFcekbbYYukEneqVUtFIq1vs1cCGwA3gJuM6z23XA+qEGKUSwaOtwccn0MQCMiomg4rgkejF0QyndpAIveMouocDTWuvXlFKfAeuUUiuAg8DXhx6mEMFhXOIIcpR7tk10RCjl9ZLoxdANOtFrrb8EZvayvQo4fyhBCREIbnp6Cy9vK+Pp78zlrMmjBtz/2c8O8t//PhuHw9G5bVdZnZkhiiAhV8YKYZK6plYA/vNfu3za/5XtRzpH815SoxdGkEQvhEnqm1qJjQxl95FjA+5b0+CeVtl1NA+S6IUxJNELYZKD1Y0syx3D6z8+l5+9uL3ffX/0rHtaZXf1zW0cqmk0K0QRJCTRC2ECl8tFfXMbI0eEMWV0LE9/epBdh+t73dc7mr/YM9umOxnVi6GSRC+ECRpa2mnvcDFyRBgAI0eEcc+/duJy9Wxp8MauI7R1uLh4elqvzyWJXgyVJHohTOA9EetN9LddNIVP91fzSi9dKV/ZfoTxiVE9TsR6yVx6MVRBtzi4EP5wsMpdVx+XEAXAN+ZOIDMllqse/4RLZpxconn/iwr+ddM5PU7EAiRFh7Ozj5KPEL6SEb0QJvCO6OM8I3qA/ImJXDpzDKXVJ59c7W80L20QhBEk0QthgvpupRuvnyyZyv0bijpv1zS0dPa26Y0kemEESfRCmOBgdSMhDlDxI07aPiZ+BNt1HRf98n3a2jv40bOF3L54Sp/PMy0tjl1l9b2exBXCV5LohTBBXVMrcSPCCAnpOVL/2cXT2HP0GL97dx8f9XKRVFfJsRG0tHVQ39xmZrgiwEmiF8IE9c2tPco2Xhdlp3J2RhK/fOuLAVeQSo6NAGSKpRiaoEv0zc3N5Ofnk52dzd133w3A/v37yczM5Oqrr6alRVb0EUNX19RKXGTvid7hcHD3pdk4QxyMT4zq93mCKdFv2l/Nt5/aRFt7h9WhBJygm14ZGRnJpk2baGxs5JxzzmHp0qVcfPHFVFZW8sknn3DppZfy+uuvWx2mGOYOVjeSNbrvxb1PS41l3/1LB3ye7LSRAOwqq+fMyYG7QI/L5eIHa7dwtP4Eu48cI0eNtDqkgBJ0I3qA9vZ2UlJSWLRoEZMnTyY+Ph6AsWPHorW2ODoRCOo9NfqhihsRSrgzhPJjzQZEZV/7Ko5z1NN7X9bJNV7QjegBnE4nhw4d4oorrqCoqOik+3o7MZaXlwdARUWFX+ITw19dU981+lMRLEsKflhc2fl1dYMkeqMF3Yi+oqKC2tpaIiIiaGtro66ujtNPPx2A++67j+9///s9HlNQUEBBQQHJybJIs/BNa7trwPq7r7LS4vpsiBYoHn7jC66ZM44QB+wtP251OAEn6BJ9WVkZCxYsYM6cOSxatIhLLrmEBx98kIyMDKqqqlixYoXVIYoAYcSIHtwnZCsDvN/N8RNtnHtaMvFR4TKiN0HQlW5mzJjB1q1bT9o2adIk9u7da1FEIlAZmeirAjz5ORxw5qQkEqMl0Zsh6Eb0QvhL3AhjxlHJsREE+oWxOWNGkhAdTqKM6E0hiV4IkxhVo88e0/c0zUDw7p5ybl88FYDJKTHsq5AavdEk0Qthktg+Lpg6VckxEYY8j119VFxJXnoCAInRYdQ0tlocUeCRRC+ECWIjQ3H20udmMLxXxwaqD/dWEhnmBCAxOmLAthDi1EmiF8IERpVtACLDnExKjjbs+ezkpc8Ps/vIsc7bGSkxFkYTuCTRC2ECo2bceAVq+eaj4kriIr86aZ0YFW5hNIFLEr0QJuirodlgBWL5xuVy8eHeSs6aPKpzW2KMJHozSKIXwgSGj+gDMNEfqGpE1zZxdmaXRC8jelNIohfCYM2t7YxPMq5GD5A9ZmTnguOB4sHXdqPiR3Dt3PGd20aEO3usyiWGThK9EAYzqnNlV8mxEVQcD6wOlh/vq+LsjKQejQQTo2VUbzRJ9EIYzKjOlV0lxwReB8u6plbOzhjVY3uCJHrDDXiNtlLqSeASoFxrnePZlgg8C6QDJcBVWusapZQDeAxYCjQC12utt5gTuhD2VFrTaOj0SoApo2N5bUcZi3PSDH1eK00dHcuyXNVje0ayTLE0mi8j+j8Di7ttuwN4W2udCbztuQ2wBMj0fKwEfm9MmEIMH+5lBI3tF+gMcVAeQCP65tZ2zullNA/uq2ObW9v9HFFgGzDRa63fB6q7bV4GrPF8vQa4vMv2v2itXVrrjUC8UipwhiBC+KCu0fjSDQyPdWNvfHoLHxQPvEBPQUnNSbNtukqMjpBVpgw22Bp9qta6DMDzOcWzXQGlXfY75NnWg1JqpVKqQClVUFVVNcgwhLCfuqY2w0/GAlQMg570r2wrY+VfNlNYWtvvfh/urSQ/PbHX+xKjw6g6LoneSEafjO2tuUevjSu01k9orfO01nlJSYG76LEIPgerGwlzGj/Poaisno5h0AfmmZVncMOfP2PeQ+/0en9tYwuPv7+P6Ijey1sZ0sHScIP9bTzqLcl4Ppd7th8CxnXZbyxwePDhCTH81DWZ032xtd1l2nMbJSEqjJnj4nnq+jlUHmvhWHPPeD/ZV9Vvf/3E6AjpSW+wwSb6l4DrPF9fB6zvsv1bSimHUuoMoM5b4hEiWNSbmIztXr5JH+VuvjZzXDy/v3YW3/vbZlraOk7a58O9lcT0MZoH91XFNZLoDTVgoldKrQU+AaYopQ4ppVYAq4FFSqliYJHnNsAG4EtgL/AH4P+YErUQNlZaY94VrHZeJLyppZ0FU1I6b8+fksI5Gcmc9rNXue+VXQCUVjfy908PcuuFp/X5PM4QB3uldGOoAeeAaa2X93HX+b3s6wJuHGpQQgxnZpZX7Dzz5kB1Q+eI3ut7503iaH0zf/hgPz+9eBof76sE6HNqpZecjDVW0C0OLoSZWts7aGwxZw54ZFiIrUs3JZUNTEw6OdE7HA7uumQaFcdPsL5Q8+HeKlJiIwbsOy/TK40liV4IA5k5mk+OjaC83r79bvZXNnJWLyP1kBAHj141k5y7XyfcGcJF2aN79LfpTk7GGkt63QhhoIPV5tXnp6XFsavMvjX6d/eU99mHPyLUyUe3L2TFvEnce0XOgM9VebyFWhnVG0YSvRAGMntEb+cafUllQ7/3p8RFcsui04gK962QIKN640iiF8JAZk6tTI6JpKaxtcd0RTtobGkzvBeP1OmNE3SJvrS0lAULFpCdnc1jjz0GwKpVq8jNzSU3N5cNGzZYHKEYzkpNLN1kj4kDYPcR+5Vv3tszcH+bU7W3XKZYGiXoTsaGhobyyCOPkJmZyezZs1m0aBEAhYWFFkcmAkFdUyuRYeaMn7zLCdqxfLN/gLLNYFRJ6cYwQZfo09LSSEtzN9TMyspCa21xRCKQmLHoiJedE31JZYOh69pGhoXI1bEGCrrSjVdJSQlbt25l7ty5AMyYMYMbbriBmpoaiyMTw5mZiT4pxr3yki0TfVXPOfRDkRQdISN6AwVlom9tbeX555/n4MGDxMXFsWrVKrZt28Zdd93FvHnzrA5PDGMHq5sMX13KKyLUSWZKDDtt1gbhWHMrn5XUcN6UZMOec3JKDPukRm+YoEv0LpeLFStWcMstt3RuKytz91174YUXyMkZeI6vEH0xY2HwrtyLhNtrRH+gyn0CeuIo40b0iVFhVMusG8MEXY3+o48+4q9//Svbtm0D4P7772ft2rV8/vnnpKen8/jjj1scoRjO6k0s3YA70W892P+iHv5WUuU+EZtuYOkmMTqCaul3Y5igS/TnnHMOrm7NsJcuXWpRNMPLksc+oOr4CV790TySYow78RYoahpaOHaijXEJ5pRuwD3Fcn2hvZZ4eHd3BalxEUzzTP80QkZKDA0t7ZTVNZE2coRhzxusgq50IwbH5XJxsKqB8mMnuO25z60Ox5a8V8WaPaK3m5KqBkNH8+BeThDk6lijSKIXPqlvbqOhpZ0pqbG8a8LFMYHAL4k+JtK05x6sksoGQ+vzAAlR7hlGNQ32XlFruJBEL3xSVtcEwE0LM7ggK5Udus7iiOynM9FHBdeIvqqhpUcf+qHyTiWtarDXiefhShK98ElBifv6gvyJifzxujwu++2H3LJOribuytu50qzplQCZKTGEhvTf4tcK8w2cWgkwcVQMDgcyxdIgkuiFTw7XNhEa4mCU5yTsTQszeX6L5oWthyyOzD7qm80v3YR0+RnYyYREY0f0zhAH8SNkiqVRJNELnxyubWL0yEicntHkDxdmMCc9gZ+9sMPiyOyjrqmV8NAQIsOcpr6O3co3aSMjGRFu/PecGB0uJ2MNIole+KTgQA15ExI6b4c6Q3jue2fxyFW5nQs/B7vS6kbGJZg/FXBamnHTGI0wv8uC4EbKSImRDpYGkUQvfHK0vpm0+J5JbHHOaP7wwX7e3VNuQVT2Ymafm66SYyNo73ANvKMf1DW2MnGUOeck3CN6mXVjBEn0wiet7S7G9JLoAaaOjuW2dZ9Tfsy+65n6g78SfUpchG0W5dhvwhx6r8TocNt8n8OdJHrhMxXf+xzu3yw/nYaWNm559nM6bDLStILfRvQx9llS0Iw59F4JUeG2eecy3Emit5nfvF1M+h2vkH7HK/xt4wGrwznJ7AmJvW7PTI2l6D8XU93QwpX/87Gfo7KPQzXmda7satqYOHbZpIPlu3vKyUyNNeW5M1JiTHneYCSJ3mZ+885eLp6RxoIpydz90k6rw+kUHe4kLrLv1kgOh4Ol00ezxWYNt/zJ5cLUzpVeo2Ls08FyoAXBhyIp2l6zi4YzSfQ20t7hIjrCyT2XZfPr5aczaVS0KUu0DcaY+BE4HP1fqLM4J81P0diXPxJ9dESoLUo3LpfL1N/PhGjzj2WwkERvE6XVjeTc/Tpbfr6IUTERxEaG8eYt53HhL/+Xb/7pU9raOyyNLy89YcB9MlJi+OHCDKpsMtq0gj9KNwA7D1vfgmJraS31zW2mPf/YhCjT1t8NNnIUbcDlcnHnC9sJcdBj1Hzf5dP5oLiSe18psig6aG5tZ4yPrWIX56Txxq6jJkdkX/44GQv2WE7QzLKNV6KnuZkYGkn0NvCPzYf4oLiSO5Zm9bjvqjnj+I9zJvLnj0v8H5jHkbre59D3Jistlld3HDE5IvsKtkRvdtudxBhJ9EYYcOERpdSTwCVAudY6x7NtFfAdwNuv9k6t9QbPfT8BVgDtwA+11q+bEHdA+cXLu8hPT+Qb+eN7vf8nS7PYV3Gcj/dWclbGKD9H525/MKaPqZXdORwOPt5bSV1jq6ldHO3KX4m+vrmN5tZ209st9Gd/VSPK5CuBE2REbwhfRvR/Bhb3sv2XWutcz4c3yU8DrgGyPY/5b6WUdb+Jw8Cr28v4/vwM1n3vTEL6GB45Qxw89e18blq7lYUPv0ddo3+vFiw4UMOs8QPX6L1SYiO4NQgXJ0mKDic6wn+LthWVWTfF0uVy8d7uchaY1P7AS6ZYGmPARK+1fh+o9vH5lgHPaK1PaK33A3uB/CHEF9BqG1v4+fqdfGfeRJ/2/59rZ1Na08iNT2/x68nZw7VNpzRyvChnNO8XV3D8hHkn6uzIHzNuujpab135psqzbOIEk66K9UqKDudEW7upr9HW3sEOXWf661hpKDX6m5RS25RSTyqlvMM9BZR22eeQZ5voxb2vFFHT2EKo07cfQ/7ERO67Yjof7q30a83+cN2ptTZYkpNGS1sH7+4Orv43/kz0Dgd8cfSY316vO++JWLP63HglRIebvsrUl5UNXPKbD9mwvczU17HSYBP974HJQC5QBjzi2d5b7aHXa5iVUiuVUgVKqYKqqqpBhjE8NZxo4+zV77DlYA0777nolB57Vd44/ufaWX6bheNyudhc4usbOrf8iYl8Y+54bv/nNpOisid/Ta0EuGTGGJ79rHTgHU3y3p4KnCEO5mUau+BIdxnJ5newfH6LJszp4JIZY0x9HSsNqqCote6cP6eU+gPwsufmIWBcl13HAr0uWa+1fgJ4AmDJkiUB19Dizx/t559bdK/31Te3criuiee+e+agTqZdkJWK8nEWzFDVN7nXij1VS3LS+PunB02IyL5GjvBffT5nTBz/+vww1Q0tJEb7/4Tl/qoGxiaMIMzHd6ODlRQTzq4yc9+5FJXVk5ESa/r3YqVBfWdKqa6XQF4BeFefeAm4RikVoZSaCGQCm4YW4vAUHRFKcmxErx+Tk2P4xbIc8tJ77x0zkFBnCN86c4JfTsYd9qwVe6rmTkokPohm3bhcLr/NuAHIUSMB6y6cKqk0r2tlVwlR4VSbfAHe7iP1ZKWZ06/HLnyZXrkWmA+MUkodAu4G5iulcnGXZUqA7wJorXcqpdYBu4A24EatdeCe4ejH1/PG8fW8cQPvOEjXzBnP/RuKePDKGaa9BrhPxA5GmDOERVmptLR1EB4auCMlr+Mn2vya6LPHuBcf2Xm43vTySW9KKhuYM8iByqmIjwqn2uRZZkfrT9huMRejDZjotdbLe9n8p372vw+4byhBiYGNjArj2YJSxidFceOCDNNep+BADWHOwV0Vc+8VOfz8xR2m/zOyg9Jq/3Su9IqPCuei7FSe/ayU75032W+v69XQ0m74guC9cYY4/LJA+BWnB/ackcAfanVTWlrKggULyM7O5rHHHgOgurqazMxMFi1aRE1NjcUR+m5e5ij++skBWk2cauldK3YwIkKdvLHriOV9evyhrqmVuEj/lqpyxoxkf2UDx5qtWYXJrD703VU1mFu6SYmNIMmGC64bKegSfWhoKI888ggbN27kd7/7Hbt27WL16tUUFxdz/vnns3r1aqtD9Nn1Z6VzpL6Z10xsOVBW20yaj31uelPT2Mqm/ac2a2c4qmtq9fs8em+d3ore9KEhDr9NCDB7euXUAC/bQBAm+rS0NGbNmkVVVRXNzc2MHTuWJ554AoA77rij8+vh4PysVO6+dBo/WLvVlOdv73Cx+WANc3zoXNmXJTmj+fGzhQG/8lRpdSPjk/xXugFYMDWFlNgIni3w/zTLeZmjfL7+Y6j2VhzH5TLn92fPkWP826zALttAECZ6r3/7t3/jV7/6FXFxA/83z8vLIy8vj4qKigH39bcrZ48lxqTL7iuOnaC9wzWkEf3inNGUHzvBloPDpyQ2GHVNrcSE+296pVeOGslO7d8RvcvlIt1PZRtwDzjqm8y5yrqorJ6po2VEH5BaW1v5xje+wde+9jUAUlNTASgrKyMlpWfvjoKCAgoKCkhO9v/shoHERoZx5eyxlNcbvzC39sy4Gcpb9IVTUwh3hgR8R8u6ptY+exWZKXtMHMXl/r1CtvzYCb/V572qTVokvOhIPZOS/fu9WCHoEr3L5WLFihXccsstndsuu+wyANasWcOyZcusCm3Qrj8rnb+ZcHFSmWcOfZqPnSt7ExsZxrzMUby244hpb7/toN6iE6LZY0bi76rYfj/Noe+qumHgRP/AhlO/Wryo7FhAXyjlFfjfYTcOh4O//OUvJ237r//6L8Bdo3/ooYesCGtI0kdF8+u3i/nn5kOGPm9BSQ2xEaFDfmv7yFUzOVrfzEOv7zEoMvs5WN1oyetelJ3q9wvT3t1Tztl+bpc90BTLV7eX8fj7X3a+C/VF5fETvP+F/cqxZgi6RB+oMlJieOrj/YaOmg/XNg1pNO8VHxXOmZOTTJ0dZLW6JmtG9A6Hg5wxI/36miWVDTj9XKaqGmBE/1ZROaEhDj790ve+WbtNbq1gJ5LoA8T1Z6WzQ9dTcMC4k56H65oYY9AUusU5o22z0LkZ6i1K9OCu07e0+e9ahZJK/757iQwLoaafGn17h4v39pSzOGc0G08h0VvZz9/fJNEHiGvPmMDKcydxzRMbT+nta3926HryJgx+amVX35g7gbXfOYNvPbkp4C6gamxpo/K4OScLfXH1nHG8uLX3BnpG26Hr2OPn9siTB+hg+eibe6htauW+y6ezruCQz207nt+q/XJ1rx1Iog8g3zpzAi6Xi79+csCw5zRqRA9w5uQk3v+igvsGcdLMzqwq23ilJ0X7rbnZDu3/JmqJ0eH9lm7eLionb0JC59KVn+4feFTf0tbB3vJjZAXBxVIgiT6gjE2I4sJpo1m7ybgZOEOZQ9+bG86eyFMflfB0ALUwtjrRh4Q42OGnq2O36zpi/bhcIrgTfU0fif5QTSO7jxzjgiz3FOm4yFA+/XLgK7H3VRyntd3F1NGB3bXSSxJ9gFl53iSOn2jj4df3GLK2rNGXud+5dCrnnZbMXet3DLzzMGHWxTynYtfhetr9MM9yx+F6spV/R8GJ0eF9Tq98x7OK2cIs9/Uv+ROTfKrT7z7i/scY6F0rvSTRB5hZ4xPYd/9Slkwfza3PfU76Ha/w+P/uo2kQi4ckx0YYfll/qDOENTfk8/at53H+I+9ZPho2glVTK7tqam3nX5/3usaPYXYdrufz0lqW54839XW6y0iJ4fiJNo70sqTlvS8XseKciUxOdi8i/uMLMimpamR9Yf/nLJ7foslRcWSmyoheDGPZY0byx+vyePHGs3ng1d3Me+hdnvpoP82tvif8MYPsWumLCUnRHKhq5Adrtw77k7N2+GcVERpiep1+h+f5vc3U/CUxyr2CVm+j+pb2Ds7P+upq9qy0OGIjQwcc1ReVHSMrCFofeEmiD3C54+J57ntnMjk5mnv+tYsFD7/n82ONPBHbm3svz+H9Lyr8tv6tWeyQ6KemxbHD5J43O3Qd0eFOJvr5qljvUom9TbGMjQw9aQEUZ4iD/PTEAev0lcdPBM2JWJBEHxTmpCfy7HfPpGT1xaz77pn84f0vB3xMbWMLsw2aWtmXa/LHs+/+pZRUNZBx5wZTX8tMpdWNjE3wT8vevlydN45PvqziYJU5ZaS29g7WFZRy9Zzxfu/pk5HiLst0n2K59WANv1iW06OFwY8uyORLH67Z+FoQdK30kkQfZMYlRvHmrqMD7qdrm/zSb9wZ4uDXy08nfVT0sL2gqq6p1a/LCPYmx3OCdIdJ5Zt9FQ00t3Z0vo4/xUeF43D0vDr27aJyzjut5zz4aWlxA84MShsZSXyU/xdVt4ok+iBUcKB6wCZRZbXNpPlpYYm4yDCevG4OK9Z8ZosyyKmqt0GiPy01ltAQh2nz3L3PO93P9XlwDwbiR4T1mGL59u5yEqJ7JutQZwhzJva/nm0wlW1AEn1Q6nB9NS2tL+72B+adjO1ufFIUB6sauenpLcPu5KwdRvSRYU4yU2PZadJ8+u26jhFhTiZ5Zrf4W0K3KZa6tqnfFgZzJyZSfqz31t0n2tqDZv68lyT6IPSNueO5a/2OfmfgFJTUkBLrv0QPsPf+pZyWGkvGT1/1+THuK4FLWPeZ/1dZ8jpY3cg4Py4M3per88byvyZ1Y3TX58f5vZmZV0a3Ngg/fWE7E/qZ+rvinIn8+u3iXu97+fMyvjZrrOEx2pkk+iC0aFoqjS3tfLyvss99vL3o/e3OpVmMoy67AAAYFElEQVTMn5LMx3v7jq2rlX/dzM/X7+TOF7azr6L/VrZmaG5t50Rbh+UjejB32mNjS7vfp1V2lRgd3rn4SGNLGx/vq2Lh1J6LBHmFOkPY2MfMm6KyetL9vOyj1STRB6EzJycRHe7kzV19l28O1xq/YpUvvCdnv//3LQOenN15uI53d5dzy6LTiAxz8uCru/0U5Ve8XSv9vTB4b7LS4nCYOOC24kSsl7cNgsvl4sPiSlraOjrbHvRlb/lxKo6d6LF995Fjflvv1i6C67sVAESEOvnF5Tl99sQpq2syrAPmYMRFhvH4N2ez6NH/5Zt/+rTH/Xf8cxvpd7zCfa8Usff+pfzw/Ew+++kFbNd1fl+E3HtV7HgblG6iI0K5bOYYU5779PHxlq6tmpESQ1uHiwNVjdz5wg4unp424OIn0eHOHuWb8vpmPvTx3WIgkUQfpBZOTemz3uprm1cznTEpifuuyOGD4pP/KA9WNfLMZ6XcuGAyf10xt3P7iHAnt104hX9tM7cNQHfeWUJ2KN2Auzd91fGeo9ih6Ohw+X1xk+68s2uqGlqoPH7ipKth+zI7PbFHJ8uiI8Gz2EhXkuiDVHxUOHPSe78gyqqyTXdXzxnPinMm8reN7rbLbxcd5ZLffMAfv5XH/71oao9/VFecrnjotT2n1OZhqLxrxdol0eeMGWn4zJuSqgZLplV2leS9OrahhRAHzJ8ycKI/Y1IiXxw9TmWXf3zBtNhIV5Log9gFWakcqOpZB7fDiN7rzqVZ3P3STm5ZV8iKNQWMS4zigmm912ZDQhzo2iae+qjEb/F5O4TGRfq3dW9fsseMNPyiqe26zu8dK7tL6NLvZtb4hM62CP2ZOzEJgE37vzopW1RWb2oPJ7uSRB/E/mPeJP7fP7b12F5woIYpNunq5wxxsPEn53PlrLGUrL6YV344r9/937rlXB5+Y4/f2iAfrG4iOtxJUkyEX15vICOjwgyfavrMplKyLS7djE0YQURoCA+8WsT9X5vu02NmT0hgXuYoHnvLXac/0dbOy9uCb2olSKIPegUHanpccXi41r8XSw0kOTaCswY48eaVkRLL8vxx/N1PC5vY4WKp7kqqGjtLSkPlcrlMa6twKhwOB0nR4dQ0tpKZ4vtFW2dMSupc+rD46HHaO1xBd1UsSKIPeu0dLt7dc/I0y7K6ZtM7V5rpxxecxogwp19eq66p1RZTK7vbZVCd/mB1I8earV9YBb46Ies4hTmkc7u0QvDW56em2ePdqj8NmOiVUuOUUu8qpYqUUjuVUj/ybE9USr2plCr2fE7wbHcopX6tlNqrlNqmlJpl9jchBm95/nh+/uIOTrS5T2B+WXGc6oYW8vo4UTscjIqJoOBnF3Dpbz40fbplaXWjLaZWdpUaF8GzBpVvnt50kDCnNVfDdjdldCwxp7iMYV56Iud43g2+sFUzc1x85yIlwcSXEX0bcKvWOgs4A7hRKTUNuAN4W2udCbztuQ2wBMj0fKwEfm941MIwF05LpaGlnY/3uaeheWfcGL1WrL9FhjnZrut4yeRVl+qb7Ve6cc+8MabcslPXM8UmfWFuvuA01tyQf8qPmzsxkZqGForK6smyyffibwMmeq11mdZ6i+frY0ARoIBlwBrPbmuAyz1fLwP+orV2aa03AvFKqTTDIxeGOHNyElHhTt7ytC72zrjxR4tis2WPieO/Xjd3uqUda/TZaiR7y4/z3p5yGlsGX3ZxuVxs13WWT630GpcYNag1Es6YnMTL2w5T09galPV5OMUavVIqHTgd+BRI1VqXgfufAeCd2KqAru8bD3m22cINN9xASspXc3BXrVqFUorc3Fw2bBi+i18MVmSYk3Mzk3mryJPo65pwOCA1zj4nYwfrpxdnoWubePKj/aY8f2t7B40t7bar0Z+bOYpQZwjXP/UZM+95g0ff/IKNX1Z1lud8daimibqmVstn3AzVjLEjeerjEiD42hN7+VzwUkrFAP8Efqy1rleqz9zdW0GvR6FUKbUSd2mHfp7LcE8++SQlJSUnbbv55pu57bbb/BaD3TzwtenMvvdNADYfqGHm2HjCQ4f/efqzJo/irVvO5aJffUBdUys/WZJl6PPbqf1BV3npiXxx75LO2/XNrWz6spoHX93T+U9v4qhoXrrpbGIj+/4ntXbTQUJDHFw5e3hPR4wIdfJlRQMqfgT5A/SpD1Q+/TUrpcJwJ/m/a62f92w+6i3JeD57p24cAsZ1efhYoEehVGv9hNY6T2udl5SUNNj4DfHb3/6WG264gZqaGkvjsEpCdDh5nnU3tc2mVg5VRkosZ0xK5IMvjO9vYrf2B32Jiwzjgmmp3HXpNDb/7AIeunIGJVUNPPrmF/0+bsfhejJTY4n00wwmswXraB58m3XjAP4EFGmtH+1y10vAdZ6vrwPWd9n+Lc/smzOAOm+Jx46+//3vs2/fPtLS0rj11lt73ScvL4+8vDwqKszp9W0HF05LpbS60T2HfpifiO0uPz2JoiPGX/peZ6POlb5KiongqrxxXDt3Ams85Yy+7NB1TLf4ilijJESF9dnyIxj4Uro5G/gmsF0pVejZdiewGlinlFoBHAS+7rlvA7AU2As0At82NGKDdXR04HQ6GTVqFPv27et1n4KCAsCd8APVf8ybRN69b9Hc2jGsp1b25rvnTeJ37+41/HlLbVq68cUvLs9h9oQE/rbxANeeMaHXfaobWrgmf7yfIzPH1rsutDoESw2Y6LXWH9J73R3g/F72dwE3DjEu0yxfvpz33nsPp9PJPffcw3vvvUdhYSETJ07k8ccftzo8S3mbPw3ni6V6ExnmZOY4408objtUR3xUmE99V+xoWe4YZt7zBhdljyY5tmcLB2eIg2lBXO4IJPboxORHa9euPen2ihUrLIrEvob7HPre5E9MpOFEG9GneMFNfz4sruTsyaMsW15vqBwOB82tHTywoYhHr87tcX9GckzA1OeD3fCfWiEMkxwbQXhoSGdL2ECSPzGJLQeNPdl+pL6ZeZm+9eCxq++eN4nnt+oey0q6XC5Llw4Uxgq6Eb3o26Y7zz+lPiLDyXmnJXPV458wLzPZsOecNCp62Newb71wCrMmJPDvf/iUktUXd25/q6ic5fnj+nmkGE5kRC86BWqS9yosrTXsKtkTbe2cM8xH814LpqSwdPpoDlY1dm7boeuYNkbq84FCEr0IGi1tHXxeWmvIc205UGvouwOr3XVJNne/tAOXy31t4w5dR1S4vOEPFJLoRdCYOjqW3xo0zfKBV4tY1MdKV8PR6JGRHKk/wVmr36GksoG3d5cP/CAxbEiiF0Fj7sRENh+oobW9Y0jPU9PQwnZt/WIcRrvvihyO1Ddz49NbrA5FGEwSvQga+ROTaGxpZ8cQk/RH+ypxmdvm3hKzxidwzZzx7DxcT4Cfrgk6kuhF0PA2tOq6WPRgfFhcSaxNFgM32u2Lp5AUHc7EUdFWhyIMFJi/rUL0Ijk2guX54/jtu3v57nmTB/UcX1Yc55nPSrn38hyDo7OH+KhwNv98kdVhCIPJiF4ElfyJiUNaA/WDYveFRecG0IwbEfgk0YugMnfi0Fpif1BcyfjEKMYnDb9GZiJ4SaIXQWVM/Ai+Nkt1zhc/VW8VHeXOpVMNjkoIc0miF0Fn7sRE9lUcH9RjQxxw5uTAuCJWBA9J9CLozJ2YxKeDnHkzc1y87VeUEqI7SfQi6ExIihrUFMu6xtaAansggockehF0HA4H6wsPn9TEyxcPvr6bHyzMMCkqIcwjiV4ErU/3V53S/h8WVxLmlD8ZMfzIb60ISglRYadUvjlQ1cDB6lN7ByCEXUiiF0HppoWZPLf5EEfqmn3a//4NRagAW0tXBA9J9CIozfX0vfGlfNPW3sHH+6qG/bKBInhJohdBKSstjpiIUJ/KN58fquNYc1vArCglgo8kehGUnCEO8tITfEr0HxZX4nDA2XKhlBimJNGLoPWDhZkUlx/ntR1lfe5zrLmVX79TzP+ZP5mE6HA/RieEcSTRi6A1XY0kMiyk36tkP9lXRXuHi3My5EIpMXxJohdBKzw0hFnj+y/ffLi3kqhwJ7MmxPsxMiGMFZQLj9xwww1s2rSJHTt2AFBdXc3cuXNJT09n3bp1JCQkWByh8JcfLMxk+R82csuzhb3e//xWzV2XTCMi1OnnyIQwTlCO6K+//vqTbq9evZri4mLOP/98Vq9ebU1QwhKnj49nTnoCnx2o7vVjSmosF+WMtjpMIYYkKEf055577km3169fz0MPPcR1113H/PnzefDBBy2KTPhbZJiT5753ltVhCGGqoBzRd3f06FEA0tLSKC8vtzgaIYQw1oAjeqXUOOAvwGigA3hCa/2YUmoV8B2gwrPrnVrrDZ7H/ARYAbQDP9Rav25C7IZJTU0FoKysjJSUFIujEUIIY/lSumkDbtVab1FKxQKblVJveu77pdb64a47K6WmAdcA2cAY4C2l1Gla63YjAzfSZZddBsCaNWtYtmyZxdEIIYSxBkz0WusyoMzz9TGlVBGg+nnIMuAZrfUJYL9Sai+QD3xiQLyGWL58OXv27GHs2LHcc8893HHHHWRmZjJ+/Hiee+45q8MTQghDndLJWKVUOnA68ClwNnCTUupbQAHuUX8N7n8CG7s87BD9/2Pwu7Vr17J27dqTthUXF1sUjRBCmMvnk7FKqRjgn8CPtdb1wO+ByUAu7hH/I55dHb083NXL861UShUopQqqqk5tAQghhBC+82lEr5QKw53k/661fh5Aa320y/1/AF723DwEjOvy8LHA4e7PqbV+AngCYMmSJT3+EdhRSUkJeXl5Voch/KiiooLkZHu2P7BrbMESV0FBgWHPZTZfZt04gD8BRVrrR7tsT/PU7wGuAHZ4vn4JeFop9Sjuk7GZwCZDo7ZIZWWl1SEIP8vLy7PtH7RdY5O47MeXEf3ZwDeB7Uop73XidwLLlVK5uMsyJcB3AbTWO5VS64BduGfs3GjnGTdCCBHofJl18yG919039POY+4D7hhCXELawcuVKq0Pok11jk7jsx+FyWV8eX7JkievVV1+1OgwhhBhWlFKbtdYDnjiUFggmee2118jIyLBdk7T09HRyc3NtcVI5JSWFnJycztuLFi0iMzOTRYsWWRjVyXGtWrUKpRS5ubnk5uZaGteCBQvIysoiOzsbcHdd9R6zmpoa28TV9Zht2NDnG3/T5efnM3PmzM649u/fz9y5c8nMzKSlpcWyuKwgid4E7e3t3HjjjezatYu1a9eya9cuq0M6SWFhoS1OSr322msn3T7//PM7u4haqXtcN998M4WFhRQW9t7K2F8eeeQRioqK2LhxI7t27WL16tWdx8zKAUX3uOCrY7Z06VLL4nrnnXf4/PPPKSwsZOPGjdx+++3cfPPNFBcX86c//cmyuKwgid4EmzZtIiMjg/DwcK655hrWr19vdUi2lJiYeNLt66677qTPVukel13MmjULgNjYWLTWrF+/vvNYvfjii7aJyy5iYmIAaG1txeFw8M4773DllVcC1h4vK0iiN4HWmnHj3JcSjB071la//A6Hg9mzZ/PEE09YHUoPaWlpJ322i9/+9rfMmDGDG264wepQAPf1HHPnzuXo0aOdx8oOXVe9ccFXx8zKklJ7ezu5ubmkpKQwefJk4uPjCQ11zz+x09+kP0iiN4FSitLSUgAOHTqEUvbpALF//342b97M4cOHefjhhwd+gB+VlZWd9NkOVq1aRUlJCdu2beOuu+6yNJbW1lYuuuginn/+eeLi4khNTe08VlZ2Xe0eV9djNm/ePMvicjqdFBYWcvz4cZ566ilqa2tpa2sDsNXfpD9IojfBnDlzKC4upqWlhWeeeaazO6bVGhoaOHbsGA0NDbzxxhsnnQi1gzVr1pz02Q66/tN54YUXLIwEVqxYQVZWFrfccgvg7rrqPVZWdl3tHlfXY2bl71htbS0ATU1NZGVlsWDBAv7xj38A1h4vS7hcLss/Fi9e7Ao0r7zyimvSpEmue++91+pQOu3bt881Y8YM17Rp02wR1+jRo12hoaEupZTrj3/8o2vhwoWujIwM18KFC20T17XXXuvKyclxTZ8+3XXppZdaGhfgmj59umvmzJmuV155xVVZWdl5zKqqqmwTV9djdvjwYcviys3NdU2fPt2VnZ3tcrncv/9z5sxxTZ482dXc3GxZXEYaM2ZMgcuHHCvz6IUQYpiSefRCCCEASfRCCBHwJNELIUSAk0QvhBABThK9EEIEOEn0QggR4CTRCyFEgJNEL4QQAU4SvRBCBDhJ9EIIEeAk0QshRICTRC+EEAFOEr0QQgQ4SfRCCBHgJNELIUSAs0U/eqVUBdAAVFodSy9GYc+4QGIbLLvGZte4QGIbLLNjm6C1Th5wL19WJ/HHh68rpUhcElugxmbXuCS24R+blG6EECLASaIXQogAZ6dE/4TVAfTBrnGBxDZYdo3NrnGBxDZYtojNFidjhRBCmMdOI3ohhBAmCLU6AKXUYuAxwAn8UWu92uJ4SoBjQDvQprXOU0olAs8C6UAJcJXWusYPsTwJXAKUa61zPNt6jUUp5cB9HJcCjcD1WustfoxrFfAdoMKz251a6w2e+34CrMB9TH+otX7djLg8rzUO+AswGugAntBaP2aT49ZXbKuw8NgppSKB94EI3DnhH1rru5VSE4FngERgC/BNrXWLUirC833MBqqAq7XWJUbHNUBsfwbOA+o8u16vtS7058+zS4xOoADQWutL7HDcurN0RO85QL8DlgDTgOVKqWlWxuSxQGudq7XO89y+A3hba50JvO257Q9/BhZ329ZXLEuATM/HSuD3fo4L4Jee45bbJVFNA64Bsj2P+W/Pz90sbcCtWuss4AzgRk8MdjhufcUG1h67E8BCrfVMIBdYrJQ6A3jQE1cmUIP7Hw6ezzVa6wzgl579zNJXbAD/t8sxK/Rs8+fP0+tHQFGX23Y4biexunSTD+zVWn+ptW7B/V9wmcUx9WYZsMbz9Rrgcn+8qNb6faDax1iWAX/RWru01huBeKVUmh/j6ssy4Bmt9Qmt9X5gL+6fuym01mXeEZzW+hjuP0CFPY5bX7H1xS/HzvO9H/fcDPN8uICFwD8827sfM++x/Adwvmckbbh+YuuL336eAEqpscDFwB89tx3Y4Lh1Z3WiV0Bpl9uH6P8X3x9cwBtKqc1KqZWebala6zJw/7ECKZZF13csdjiWNymltimlnlRKJVgdl1IqHTgd+BSbHbdusYHFx04p5VRKFQLlwJvAPqBWa93Wy2t3xuW5vw5IMiOu3mLTWnuP2X2eY/ZLT1nkpNh6idsMvwL+H+5SHLiPgy2OW1dWJ/re/ptZPQ3obK31LNxvAW9USp1rcTy+svpY/h6YjPvtdRnwiGe7JXEppWKAfwI/1lrX97Or3+PrJTbLj53Wul1rnQuMxf2uIauf1/brMesem1IqB/gJMBWYg7sWfru/Y1NKec9Tbe6yub/Xt+xv1OpEfwgY1+X2WOCwRbEAoLU+7PlcDryA+5f+qPftn+dzuXUR9hmLpcdSa33U8wfZAfyBr0oMfo9LKRWGO5H+XWv9vGezLY5bb7HZ6dhprWuB93CfQ4hXSnknbHR97c64PPePxPdSnhGxLfaUwVxa6xPAU1hzzM4GLvNM4HgGd8nmV9jsuIH1if4zIFMpNVEpFY77xNNLVgWjlIpWSsV6vwYuBHZ4YrrOs9t1wHprIoR+YnkJ+JZSyuE5WVXnLVX4Q7c66BW4j5s3rmuUUhGe2QiZwCYT43AAfwKKtNaPdrnL8uPWV2xWHzulVLJSKt7z9QjgAtznD94FrvTs1v2YeY/llcA7WmuzRs29xba7yz9tB+4aeNdj5pefp9b6J1rrsVrrdNy56x2t9TewwXHrztLplVrrNqXUTcDruKdXPqm13mlhSKnAC0opcB+bp7XWrymlPgPWKaVWAAeBr/sjGKXUWmA+MEopdQi4G1jdRywbcE8p24t7Wtm3/RzXfKVULu63oiXAdwG01juVUuuAXbhnndyotW43Kzbco6xvAts9dV2AO7HBcesntuUWH7s0YI1nRk8IsE5r/bJSahfwjFLqXmAr7n9SeD7/VSm1F/eI9BoTYhootneUUsm4yyGFwPc8+/vz59mX27H+uJ1ErowVQogAZ3XpRgghhMkk0QshRICTRC+EEAFOEr0QQgQ4SfRCCBHgJNELIUSAk0QvhBABThK9EEIEuP8PjI+sbHi+HVkAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1125ab9e8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.imshow(img, cmap='Set3')  # show me the sample image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 167,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZYAAAF9CAYAAAAne07XAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJzt3X+UnmV95/F3mghVWqBOtYufQEGJttS2oaTYc1w87bIqY5XgSiVoLbSslNbUUnddf9T2ANvuAtICXVh3+aVAi1BZrbE1ixXsj92jLglSFCg1ZNOSL6zoBCnUHzQ4+8d9Dzw+zGSeTK6Zycj7dc6cee7r/t7XXPfDEz5z/7pm2eTkJJIktfJdiz0ASdJ3FoNFktSUwSJJaspgkSQ1ZbBIkpoyWCRJTa1Y7AEshCTvBV4DPAbcC/xiVX11mrqrgFcDD1bViwfazwLeDHy5b3p3VX18YP0hwF3AWVV1wS7G8SzgQ8ALgMeBj1XVO/ds7yRp7/Idd8SS5KeTfGCo+c+BF1fVjwF/B7xrhs0/ABw3w7oLq2p1//Xx4XXAxhGHeEFV/RBwJPDSJOMjbidJS8LT4oilqj4xsPgZ4MQZ6v4qyaG703eSE4CtwD8Ntf888FZgH+CzwK9W1deAT/U/67EktwErd+fnSdLe7jvuiGUEv8ToRxeD1ie5I8lVSb4PIMl+wDuAswcLk/wwcBLw0qpaTXfa641DNQfSnZ67eQ5jkaS91nfMEUuSzwL7At8DPDvJ7f2qd1TVTX3NbwI7gT/aze7fB/xHYLL//nt0AXU23SmyR5MM1h8LHAXc2rc/E3hwYKwrgA8Cf1BVW3dzLJK0V/uOCZaqegl011iAU6vq1MH1SU6huzB/bFXt1gRpVfWlgX4uB/60X3wJcGKS84EDgW8l+QawDLi6qma6lnMZ8MWqumh3xiFJS8HT4lRYkuPoTlkd31/n2N3tDxpYfC3wBYCqOqaqDq2qQ4GLgP9UVZfQnd46Mclz++2fneQH+9e/AxwAnLkHuyRJe62nRbAAlwDfC/x5ktuT/DeAJM9LMnjb8AeBTwMvSrI9yWn9qvOTfD7JHcDPAL+xqx9WVXcB7wE+0W/z58BBSVYCvwkcAdzWj+Xftt1VSVpcy5w2X5LU0tPliEWStEBGunjfX6O4GFgOXFFV5w6t3xe4hu5OqAngpKra1j8TcjdwT1/6mao6I8n3An890MVK4A+r6swkpwLvBapfd0lVXTGXnZMkLbxZgyXJcuBS4OXAdrpbaDf01xGmnAY8VFWHJ1kHnEf3HAfAvf2zHE+oqkeAJ9qSbAY+PFByQ1WtH3UnXvziF08efPDBo5ZLkoA77rjjK1X1nNb9jnLEcjSwZep5iyTXA2vp5saashY4q399I3BJkmWjDCDJKuC5fPsRzG45+OCD2bhxLs88StLTV5K/n49+RwmWAPcNLG+ne35j2pqq2pnkYWCsX3dYks8B/wi8p6qGA+RkuiOUwbsIXpfkZXTzev1GVd03tA1JTgdO71+PsBuSpIUwysX76Y48hm8lm6nmAeCQqjoSeBtwXZL9h+rW0T2FPuVjwKH9hJGfBK6eblBVdVlVramqNWNjY9OVSJIWwSjBsh0YvICxErh/ppp+upIDgB1V9c2qmgCoqs10U9a/cGqjJD8OrOjX0ddNVNU3+8XL6W4IkCQtEaMEy63AqiSHJdmH7ghjw1DNBuCU/vWJwC1VNZnkOf3Ff5I8H1hFNxPwlJP59qOV4afcj6e7q0yStETMeo2lv2ayHriJ7nbjq6rqziTnAJuqagNwJXBtki3ADrrwAXgZcE6SnXQz/J5RVTsGun898KqhH/nWJMfTTRa5Azh1znsnSVpw3xFP3o+Pj096V5gk7Z4km6tqTet+ffJektSUwSJJaspgkSQ1ZbBIkpoyWCRJTRkskqSmDBZJUlMGiySpKYNFktSUwSJJaspgkSQ1ZbBIkpoyWCRJTRkskqSmDBZJUlMGiySpKYNFktSUwSJJaspgkSQ1ZbBIkpoyWCRJTRkskqSmDBZJUlMGiySpKYNFktSUwSJJaspgkSQ1ZbBIkpoyWCRJTRkskqSmDBZJUlMrRilKchxwMbAcuKKqzh1avy9wDXAUMAGcVFXbkhwK3A3c05d+pqrO6Lf5C+Ag4Ov9uldU1YMz9TXXHZQkLaxZgyXJcuBS4OXAduDWJBuq6q6BstOAh6rq8CTrgPOAk/p191bV6hm6f2NVbRpq21VfkqS93Cinwo4GtlTV1qp6DLgeWDtUsxa4un99I3BskmVzHFPLviRJC2yUYAlw38Dy9r5t2pqq2gk8DIz16w5L8rkkf5nkmKHt3p/k9iS/NRAeu+pLkrSXGyVYpjtamByx5gHgkKo6EngbcF2S/fv1b6yqHwWO6b/etBs/jySnJ9mUZNPExMQIuyFJWgijBMt24OCB5ZXA/TPVJFkBHADsqKpvVtUEQFVtBu4FXtgvV//9EeA6ulNuM/Y1PKiquqyq1lTVmrExD2gkaW8xSrDcCqxKcliSfYB1wIahmg3AKf3rE4FbqmoyyXP6i/8keT6wCtiaZEWS7+/bnwG8GvjCrvqa2+5JkhbarHeFVdXOJOuBm+huN76qqu5Mcg6wqao2AFcC1ybZQnd0sa7f/GXAOUl2Ao8DZ1TVjiT7ATf1obIc+CRweb/NTH1JkpaAZZOTS/9gYHx8fHLjxo2LPQxJWlKSbK6qNa379cl7SVJTBoskqSmDRZLUlMEiSWrKYJEkNWWwSJKaMlgkSU0ZLJKkpgwWSVJTBoskqSmDRZLUlMEiSWrKYJEkNWWwSJKaMlgkSU0ZLJKkpgwWSVJTBoskqSmDRZLUlMEiSWrKYJEkNWWwSJKaMlgkSU0ZLJKkpgwWSVJTBoskqSmDRZLUlMEiSWrKYJEkNWWwSJKaMlgkSU2tGKUoyXHAxcBy4IqqOndo/b7ANcBRwARwUlVtS3IocDdwT1/6mao6I8mzgA8BLwAeBz5WVe/s+zoVeC9Q/TaXVNUVc95DSdKCmjVYkiwHLgVeDmwHbk2yoaruGig7DXioqg5Psg44DzipX3dvVa2epusLqupTSfYBbk4yXlUb+3U3VNX6ue6UJGnxjHIq7GhgS1VtrarHgOuBtUM1a4Gr+9c3AscmWTZTh1X1tar6VP/6MeA2YOXuDl6StPcZJVgC3DewvL1vm7amqnYCDwNj/brDknwuyV8mOeYpnScHAq8Bbh5ofl2SO5LcmOTg0XZFkrQ3GCVYpjvymByx5gHgkKo6EngbcF2S/acKkqwAPgj8QVVt7Zs/BhxaVT8GfJInj4S+TZLTk2xKsmliYmKE3ZAkLYRRgmU7MHjUsBK4f6aaPiwOAHZU1TeragKgqjYD9wIvHNjuMuCLVXXRVENVTVTVN/vFy+luCHiKqrqsqtZU1ZqxsbHpSiRJi2CUYLkVWJXksP5C+zpgw1DNBuCU/vWJwC1VNZnkOf3Ff5I8H1gFbO2Xf4cugM4c7CjJQQOLx9PdVSZJWiJmvSusqnYmWQ/cRHe78VVVdWeSc4BNVbUBuBK4NskWYAdd+AC8DDgnyU6624rPqKodSVYCvwn8LXBbEnjytuK3Jjke2Nn3dWq73ZUkzbdlk5PDl0uWnvHx8cmNGzfOXihJekKSzVW1pnW/PnkvSWrKYJEkNWWwSJKaMlgkSU0ZLJKkpgwWSVJTBoskqSmDRZLUlMEiSWrKYJEkNWWwSJKaMlgkSU0ZLJKkpgwWSVJTBoskqSmDRZLUlMEiSWrKYJEkNWWwSJKaMlgkSU0ZLJKkpgwWSVJTBoskqSmDRZLUlMEiSWrKYJEkNWWwSJKaMlgkSU0ZLJKkpgwWSVJTBoskqSmDRZLU1IpRipIcB1wMLAeuqKpzh9bvC1wDHAVMACdV1bYkhwJ3A/f0pZ+pqjP6bY4CPgA8E/g48OtVNZnk2cANwKHANuD1VfXQ3HdRkrSQZj1iSbIcuBQYB44ATk5yxFDZacBDVXU4cCFw3sC6e6tqdf91xkD7+4DTgVX913F9+zuBm6tqFXBzvyxJWiJGORV2NLClqrZW1WPA9cDaoZq1wNX96xuBY5Msm6nDJAcB+1fVp6tqku5o54Rp+rp6oF2StASMEiwB7htY3t63TVtTVTuBh4Gxft1hST6X5C+THDNQv32GPn+gqh7o+3oAeO60g0pOT7IpyaaJiYkRdkOStBBGCZbpjjwmR6x5ADikqo4E3gZcl2T/Efvcpaq6rKrWVNWasbGx2TeQJC2IUYJlO3DwwPJK4P6ZapKsAA4AdlTVN6tqAqCqNgP3Ai/s61fO0OeX+lNlU6fMHtydHZIkLa5RguVWYFWSw5LsA6wDNgzVbABO6V+fCNzS3+H1nP7iP0meT3eRfmt/iuuRJD/VX4v5BeCj0/R1ykC7JGkJmDVY+msm64Gb6G4d/uOqujPJOUmO78uuBMaSbKE75TV1J9fLgDuS/A3dRf0zqmpHv+5XgCuALXRHMhv79nOBlyf5IvDyflmStEQsm5zcrUsbe6Xx8fHJjRs3zl4oSXpCks1VtaZ1vz55L0lqymCRJDVlsEiSmjJYJElNGSySpKYMFklSUwaLJKkpg0WS1JTBIklqymCRJDVlsEiSmjJYJElNGSySpKYMFklSUwaLJKkpg0WS1JTBIklqymCRJDVlsEiSmjJYJElNGSySpKYMFklSUwaLJKkpg0WS1JTBIklqymCRJDVlsEiSmjJYJElNGSySpKYMFklSUwaLJKmpFaMUJTkOuBhYDlxRVecOrd8XuAY4CpgATqqqbQPrDwHuAs6qqguSvAi4YaCL5wO/XVUXJTkLeDPw5X7du6vq43PYN2nR/MnnivfedA/3f/XrPO/AZ/L2V76IE47MYg9LWhCzBkuS5cClwMuB7cCtSTZU1V0DZacBD1XV4UnWAecBJw2svxDYOLVQVfcAqwf6L+Ajg/VVdcHcdklaXH/yueJdH/48X//nxwGor36dd3348wCGi54WRjkVdjSwpaq2VtVjwPXA2qGatcDV/esbgWOTLANIcgKwFbhzhv6PBe6tqr/f3cFLe6P33nTPE6Ey5ev//DjvvemeRRqRtLBGCZYA9w0sb+/bpq2pqp3Aw8BYkv2AdwBn76L/dcAHh9rWJ7kjyVVJvm/aQSWnJ9mUZNPExMQIuyEtjPu/+vXdape+04wSLMumaZscseZsutNaj07XcZJ9gOOBDw00vw94Ad2psgeA35tu26q6rKrWVNWasbGxXe+BtICed+Azd6td+k4zSrBsBw4eWF4J3D9TTZIVwAHADuAlwPlJtgFnAu9Osn5gu3Hgtqr60lRDVX2pqh6vqm8Bl9OdipOWjLe/8kU88xnLv63tmc9Yzttf+aJFGpG0sEa5K+xWYFWSw+gusq8D3jBUswE4Bfg0cCJwS1VNAsdMFfR3ez1aVZcMbHcyQ6fBkhxUVQ/0i68FvjDy3kh7gakL9N4VpqerWYOlqnb2Rxk30d1ufFVV3ZnkHGBTVW0ArgSuTbKF7khl3Wz9JnkW3Z1mvzy06vwkq+lOpW2bZr201zvhyBgketpaNjk5fLlk6RkfH5/cuHHj7IWSpCck2VxVa1r365P3kqSmDBZJUlMGiySpKYNFktSUwSJJaspgkSQ1ZbBIkpoyWCRJTRkskqSmDBZJUlMGiySpKYNFktSUwSJJaspgkSQ1ZbBIkpoyWCRJTRkskqSmDBZJUlMGiySpKYNFktSUwSJJaspgkSQ1ZbBIkpoyWCRJTRkskqSmDBZJUlMGiySpKYNFktSUwSJJaspgkSQ1ZbBIkppaMUpRkuOAi4HlwBVVde7Q+n2Ba4CjgAngpKraNrD+EOAu4KyquqBv2wY8AjwO7KyqNX37s4EbgEOBbcDrq+qhue6gJGlhzXrEkmQ5cCkwDhwBnJzkiKGy04CHqupw4ELgvKH1FwIbp+n+Z6pq9VSo9N4J3FxVq4Cb+2VJ0hIxyqmwo4EtVbW1qh4DrgfWDtWsBa7uX98IHJtkGUCSE4CtwJ0jjmmwr6uBE0bcTpK0FxglWALcN7C8vW+btqaqdgIPA2NJ9gPeAZw9Tb+TwCeSbE5y+kD7D1TVA31fDwDPHWVHJEl7h1GCZdk0bZMj1pwNXFhVj06z/qVV9RN0p9jekuRlI4zlCUlOT7IpyaaJiYnd2VSSNI9GCZbtwMEDyyuB+2eqSbICOADYAbwEOL+/UH8m8O4k6wGq6v7++4PAR+hOuQF8KclBfV8HAQ9ON6iquqyq1lTVmrGxsRF2Q5K0EEa5K+xWYFWSw4AC1gFvGKrZAJwCfBo4EbilqiaBY6YKkpwFPFpVl/SnyL6rqh7pX78COGeor3P77x+d475JkhbBrMFSVTv7o4yb6G43vqqq7kxyDrCpqjYAVwLXJtlCd6SybpZufwD4SJKpMVxXVf+zX3cu8MdJTgP+Afi5OeyXJGmRLJucHL5csvSMj49Pbtw43d3MkqSZJNk89LhHEz55L0lqymCRJDVlsEiSmjJYJElNGSySpKYMFklSUwaLJKkpg0WS1JTBIklqymCRJDVlsEiSmjJYJElNGSySpKYMFklSUwaLJKkpg0WS1JTBIklqymCRJDVlsEiSmjJYJElNGSySpKYMFklSUwaLJKkpg0WS1JTBIklqymCRJDVlsEiSmjJYJElNGSySpKYMFklSUwaLJKkpg0WS1NSKUYqSHAdcDCwHrqiqc4fW7wtcAxwFTAAnVdW2gfWHAHcBZ1XVBUkO7uv/BfAt4LKqurivPQt4M/DlfvN3V9XH57qDkqSFNesRS5LlwKXAOHAEcHKSI4bKTgMeqqrDgQuB84bWXwhsHFjeCfy7qvph4KeAtwz1eWFVre6/DBVJWkJGORV2NLClqrZW1WPA9cDaoZq1wNX96xuBY5MsA0hyArAVuHOquKoeqKrb+tePAHcD2ZMdkSTtHUYJlgD3DSxv56kh8ERNVe0EHgbGkuwHvAM4e8bOk0OBI4HPDjSvT3JHkquSfN8M252eZFOSTRMTEyPshiRpIYwSLMumaZscseZsutNaj07XcZLvAf4HcGZV/WPf/D7gBcBq4AHg96bbtqouq6o1VbVmbGxs9r2QJC2IUS7ebwcOHlheCdw/Q832JCuAA4AdwEuAE5OcDxwIfCvJN6rqkiTPoAuVP6qqD091VFVfmnqd5HLgT3d/tyRJi2WUYLkVWJXkMKCAdcAbhmo2AKcAnwZOBG6pqkngmKmC/m6vR/tQWQZcCdxdVb8/2FGSg6rqgX7xtcAXdnuvJEmLZtZgqaqdSdYDN9HdbnxVVd2Z5BxgU1VtoAuJa5NsoTtSWTdLty8F3gR8PsntfdvUbcXnJ1lNdyptG/DLc9gvSdIiWTY5OXy5ZOkZHx+f3Lhx4+yFkqQnJNlcVWta9+uT95KkpgwWSVJTBoskqSmDRZLUlMEiSWrKYJEkNWWwSJKaMlgkSU0ZLJKkpgwWSVJTBoskqSmDRZLUlMEiSWrKYJEkNWWwSJKaMlgkSU0ZLJKkpgwWSVJTBoskqSmDRZLUlMEiSWrKYJEkNWWwSJKaMlgkSU0ZLJKkpgwWSVJTBoskqSmDRZLUlMEiSWrKYJEkNWWwSJKaWjFKUZLjgIuB5cAVVXXu0Pp9gWuAo4AJ4KSq2jaw/hDgLuCsqrpgV30mOQy4Hng2cBvwpqp6bA/2UZK0gGY9YkmyHLgUGAeOAE5OcsRQ2WnAQ1V1OHAhcN7Q+guBjSP2eR5wYVWtAh7q+5YkLRGjnAo7GthSVVv7I4frgbVDNWuBq/vXNwLHJlkGkOQEYCtw52x99tv8q74P+j5P2P3dkiQtllGCJcB9A8vb+7Zpa6pqJ/AwMJZkP+AdwNkj9jkGfLXvY6af1XWQnJ5kU5JNExMTI+yGJGkhjBIsy6Zpmxyx5my601qPjlg/ys8CoKouq6o1VbVmbGxsuhJJ0iIY5eL9duDggeWVwP0z1GxPsgI4ANgBvAQ4Mcn5wIHAt5J8A9g8Q59fAQ5MsqI/apnuZ0mS9mKjBMutwKr+bq0C1gFvGKrZAJwCfBo4EbilqiaBY6YKkpwFPFpVl/Th85Q+q2oyyaf6Pq7v+/zoHuyfJGmBzXoqrD9yWA/cBNwN/HFV3ZnknCTH92VX0l1T2QK8DXjnXPrsV78DeFvf11jftyRpiVg2OTntJYwlZXx8fHLjxo2zF0qSnpBkc1Wtad2vT95LkpoyWCRJTRkskqSmDBZJUlMGiySpKYNFktSUwSJJaspgkSQ1ZbBIkpoyWCRJTRkskqSmDBZJUlMGiySpKYNFktSUwSJJaspgkSQ1ZbBIkpoyWCRJTRkskqSmDBZJUlMGiySpKYNFktSUwSJJaspgkSQ1ZbBIkpoyWCRJTRkskqSmDBZJUlMGiySpKYNFktSUwSJJamrFKEVJjgMuBpYDV1TVuUPr9wWuAY4CJoCTqmpbkqOBy/qyZcBZVfWRJC8Cbhjo4vnAb1fVRUnOAt4MfLlf9+6q+vic9k6StOBmDZYky4FLgZcD24Fbk2yoqrsGyk4DHqqqw5OsA84DTgK+AKypqp1JDgL+JsnHquoeYPVA/wV8ZKC/C6vqggb7J0laYKOcCjsa2FJVW6vqMeB6YO1QzVrg6v71jcCxSZZV1deqamff/t3A5DT9HwvcW1V/v/vDlyTtbUYJlgD3DSxv79umremD5GFgDCDJS5LcCXweOGMgaKasAz441LY+yR1JrkryfSPtiSRprzBKsCybpm34yGPGmqr6bFX9CPCTwLuSfPdUQZJ9gOOBDw1s9z7gBXSnyh4Afm+6QSU5PcmmJJsmJiZG2A1J0kIY5eL9duDggeWVwP0z1GxPsgI4ANgxWFBVdyf5J+DFwKa+eRy4raq+NFD3xOsklwN/Ot2gquoy+hsDknw5yd5yKu37ga8s9iBGtFTGulTGCUtnrEtlnLB0xrpUxglPjvUH56PzUYLlVmBVksPoLrKvA94wVLMBOAX4NHAicEtVTfbb3NdfvP9B4EXAtoHtTmboNFiSg6rqgX7xtXQ3AOxSVT1nhP1YEEk2VdWaxR7HKJbKWJfKOGHpjHWpjBOWzliXyjhh/sc6a7D0obAeuInuduOrqurOJOcAm6pqA3AlcG2SLXRHKuv6zf8l8M4k/wx8C/jVqvoKQJJn0d1p9stDP/L8JKvpTqVtm2a9JGkvNtJzLP1zJB8favvtgdffAH5umu2uBa6doc+v0V/gH2p/0yhjkiTtnXzyvr3LZi/ZayyVsS6VccLSGetSGScsnbEulXHCPI912eTkdI+WSJI0Nx6xSJKaGukay9PNXOdG69e9i26Km8eBt1bVTX37NuCRvn3n1B0ZSZ5NN2/aoXQ3K7y+qh5ajHHO5xxuezDf3BjdbA4/CXygqtYPbHMU8AHgmXTXAH+9vxtxwd/TmcbZ36TyIbpnsx4HPlZV7+zXnQq8l+5uS4BLquqKUcY5H2Ptt/kL4CDg633TK6rqwV19lhZ6nEm+F/jrgS5WAn9YVWfuyXu6B+N8OXAusA/wGPD2qrql36b5Z3Q+xtr6c+oRy5CBudHGgSOAk5McMVT2xNxowIV0c6PR160DfgQ4DvivfX9TfqaqVg/d5vdO4OaqWgXc3C8vyjir6p5+fKvpPpBf46lzuK3uv3YnVOY8VuAbwG8B/36art8HnA6s6r+O69sX/D2dZZwXVNUPAUcCL00yPrDuhoH3dHdCZb7GCvDGgTE9OEtfCz7OqnpkYHyrgb8HPjxQstvv6R6O8yvAa6rqR+keuxi8YanpZ3Sex9rsc2qwPNWc50br26+vqm9W1f8FtvT97cpgX1cDJ+wl42w5h9uezDf3T1X1v+j+J/OEflLT/avq01U1Sffb2QnT9LUg7+lM46xuvrxP9a8fA26j+w17TzUf6yxm+iwt6jiTrAKey7cfwczFnozzc1U19dD4ncB3J9l3nj6j8zLW1p9Tg+Wp9mRutF1tOwl8IsnmJKcP1PxA9Q+E9t+fu8jjnNJyDrc9mm9uF31un6HPxXhPZ+88ORB4Dd1vqFNe17+nNyY5eIZNF3qs709ye5LfGgiPufY1r+8p3UPWN/T/454yl/e01ThfB3yuqr7J/HxG52usT27Y4HNqsDzVnsyNtqttX1pVP0F3+PqWJC+b+xB3OYZRana57Z7M4TYPY92TPnfXfIwTgH6qow8Cf1BVW/vmjwGHVtWPAZ/kyd8wF3Osb+xPkxzTf009VzbX93ve3tPe8C9Ac31P93icSX6E7pTTL49SvwfmY6xT7U0+pwbLU+3O3GhT/yGm5kabcdupw8/+nPVHePLU05f6Q+ap0zsPMpp5GWdv2jncqurxqvoWcDmzn+JrNdZd9Tl4qD7Y52K8p7O5DPhiVV001VBVEwO/LV5Od11rVPMy1qqq/vsjwHU8+d95rvs9b+9pkh8HVlTV5oHxz/U93aNxJllJ9+/6F6rq3oH61p/R+RrrlCafU4PlqZ6YG63/zX0d3Vxog6bmRoOBudH69nX9+dXD6C7W/Z8k+/V3spBkP+AVPDkH2mBfpwAfXaxxDmw37RxuA4sjzeHWaKzT6k8fPJLkp/rTNb/Ak+/dYrynM0ryO3T/sM8cah98T48H7h5xnPMy1iQrknx///oZwKuZ/nM60n7P1zgHzPY53Z33dM7j7E8d/Rnwrqr631PF8/QZnZexQtvPqQ9ITiPJq4CLeHJutN/NwNxo6ab+v5bu7okdwLqpw8Ykvwn8ErATOLOqNiZ5Pk/eXbUCuK6qfrevHwP+GDgE+Afg56pqlN8Em4+zb38W3bnZ51fVwwM/61q602BPzOFWT04WOt9j3QbsT3eL5FfpboG9K8kanryVcyPwa/0/nsV6T58yTuAf+/fzb4Gp3/ouqaorkvxnun+oO/u+fqWq/nax3lO6u6v+CnhG3+cngbdV1eO76muhx1n9X69NshV41eB7tifv6VzHmeQ9wLuALw50N3WbdvPP6HyMle79bfY5NVgkSU15KkyS1JTBIklqymCRJDVlsEiSmnISSkmaB0l+DjgL+GHg6KratIva5cAmukeJXt23XQmsoXvY8e+AU6vq0SRnAG+hmyzyUeD0qTvlZuh7Nd0Dzvv32/xuVd0wU31yoUr+AAACLUlEQVQLHrFI0h5K8tNJPjDU/AXg39Ddwj2bX+epz4f8RlX9eP/E+z8AU7NQX1dVP1rdBJznA78/S99fo3sYcmrS2Yv651nmjUcskjQPqupugGR4Gq9v1z8J/7PA7wJvG9j+H/v1y+ieg5kcbO/tN9XeH/WcC/w0sC9waVX996r6u4E+70/yIPAcuueC5oVHLJK0uC4C/gPwreEVSd4P/D/gh4D/MtD+liT30h2xvLVvPg14uKp+ku5v2Ly5n1ljsL+j6R6GHJ7KpSmDRZLmKMlnk9wOXAEcn25W6NuTvHLE7V8NPDg439mgqvpF4Hl0p8lOGmi/tKpeALwDeE/f/ArgF/rxfJZuNuNVAz/rILqn8X+xn/Nv3ngqTJLmqKpeAt01FrqL66fuZhcvpQukVwHfDeyf5A+r6ucHfsbjSW4A3g68f2j76+kuzEN3kf/Xqv+rtYOS7E83R9h7quozuznG3eYRiyQtkqp6V1WtrKpD6SaTvKWqfj7JsiSHwxPXWF5DN4/X1B83m/KzPDnv103Ar/QTiJLkhf0EuPvQzVV4TVUN/imMeeMRiyTNgySvpbsu8hzgz5LcXlWvTPI8ur9T/6pdbL4MuLo/0lgG/A3wK/269Un+NfDPwEM8OYvxFcChwG19GH2Z7i9TvhZ4GTCW7u/XQ3d0dXubPZ1m8E5CKUlqyVNhkqSmDBZJUlMGiySpKYNFktSUwSJJaspgkSQ1ZbBIkpoyWCRJTf1/Z0NWd0Aef6wAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1125abdd8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(cx, cy)           # show me its center\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 174,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.cross_validation import ShuffleSplit\n",
    "from sklearn.grid_search import GridSearchCV\n",
    "def fit_model(X,y):\n",
    "    cv_sets = ShuffleSplit(X_train.shape[0],n_iter=10,test_size=0.20,random_state=1234)\n",
    "    rf = RandomForestClassifier()\n",
    "    params = {'criterion':('gini','entropy'),\n",
    "              'max_depth':[2,3],\n",
    "              'min_samples_split':[30,10],\n",
    "             'max_features' : [4,5],\n",
    "             'n_estimators' : [50,100]}\n",
    "    grid = GridSearchCV(rf,params,cv=cv_sets)\n",
    "    grid = grid.fit(X,y)\n",
    "    return grid.best_estimator_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 175,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RandomForestClassifier(bootstrap=True, class_weight=None, criterion='entropy',\n",
       "            max_depth=3, max_features=4, max_leaf_nodes=None,\n",
       "            min_impurity_decrease=0.0, min_impurity_split=None,\n",
       "            min_samples_leaf=1, min_samples_split=10,\n",
       "            min_weight_fraction_leaf=0.0, n_estimators=100, n_jobs=1,\n",
       "            oob_score=False, random_state=None, verbose=0,\n",
       "            warm_start=False)"
      ]
     },
     "execution_count": 175,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fit_model(X_train,y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 176,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "nsimu = 21\n",
    "accuracy = [0]*nsimu\n",
    "ntree = [0]*nsimu\n",
    "for i in range(1,nsimu):\n",
    "    rf = RandomForestClassifier(n_estimators=i*5,min_samples_split=10,max_depth=None,criterion='gini')\n",
    "    rf.fit(X_train,y_train)\n",
    "    rf_pred = rf.predict(X_test)\n",
    "    cm = confusion_matrix(y_test,rf_pred)\n",
    "    accuracy[i] = (cm[0,0]+cm[1,1])/cm.sum()\n",
    "    ntree[i]=i*5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 177,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA4AAAAGfCAYAAAAOK7wwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJzt3X+wnfV9J/a3VkIk3sTgKOuU+5FaSGA3xU5Miky8Tbezs6wpN42Rd0yMGDeGLVuYnZUdt0089u7QH56dKThpHXbNeIqBGNiscaKNG7mJyoxNdppkWhthHDfYyURmFUsfXBNfCGs7G4PS2z/OkTlcS9yDDnCFvq/XzJn7PN/zeb7388ycOdL7fp/znE2rq6sBAADg9PdXNroBAAAAXhoCIAAAwCAEQAAAgEEIgAAAAIMQAAEAAAYhAAIAAAxCAAQAABiEAAgAADAIARAAAGAQWza6gRfCa1/72tUdO3ZsdBsAAAAb4vOf//zXuvuvrVd3WgTAHTt2ZP/+/RvdBgAAwIaoqj+Zp84loAAAAIMQAAEAAAYhAAIAAAxCAAQAABiEAAgAADAIARAAAGAQAiAAAMAgBEAAAIBBCIAAAACDEAABAAAGsWWeoqq6PMktSTYnub27b1rz/JlJ7k5ycZKVJFd196GqemOSm5JsTfJUkp/v7vur6nuT/M7MFNuT/IvufldVXZvkF5L09LkPdvftJ3uCAAAATKwbAKtqc5Jbk7wxyZEkD1TVvu7+wkzZdUme6O7zq2p3kpuTXJXka0ne1N2PVtVrk9yXpLr760kumvkdDyb59Zn5PtbdexY8NwAAAGbMcwnoJUkOdvcj3f1UknuT7FpTsyvJXdPtvUkurapN3f1Qdz86HX84yXdNVwu/raouSPLqPHtFEAAAgBfYPAGwkhye2T8yHTtuTXcfTfJkkm1rat6S5KHu/taa8aszWfFbna2tqs9X1d6q2jFHjwAAAKxjngC46Thjq8+npqpek8lloTccp253ko/O7H8iybnd/aNJPplnVhafpaqur6oDVXVgZWXlOdoHAAAgmS8AHkkyuwq3PcmjJ6qpqi1Jzkry+HR/e5KPJ3l7d39p9qCqel2SLd394LGx7l6ZWSX8cCY3lvkO3X1bd+/s7p3btq1dbAQAAGCteQLgA0kuqKrzqmprJit2+9bU7EtyzXT7yiT3d/dqVZ2d5DeTvLe7f+84c1+dZ6/+parOmdm9IskX5+gRAACAdax7F9DuPlpVezK5g+fmJHd298NV9b4kB7p7X5I7ktxTVQczWfnbPT18T5Lzk9xYVTdOxy7r7sem229N8pNrfuU7q+qKJEenc1170mcHAADAt21aXV37cb6Xn+Xl5dX9+/dvdBsAAAAboqoe7O6d69XNcwkoAAAApwEBEAAAYBACIAAAwCAEQAAAgEEIgAAAAIMQAAEAAAYhAAIAAAxCAAQAABiEAAgAADAIARAAAGAQAiAAAMAgBEAAAIBBCIAAAACDEAABAAAGIQACAAAMQgAEAAAYhAAIAAAwCAEQAABgEAIgAADAIARAAACAQQiAAAAAgxAAAQAABiEAAgAADEIABAAAGIQACAAAMAgBEAAAYBACIAAAwCAEQAAAgEEIgAAAAIMQAAEAAAYhAAIAAAxCAAQAABiEAAgAADAIARAAAGAQAiAAAMAgBEAAAIBBbJmnqKouT3JLks1Jbu/um9Y8f2aSu5NcnGQlyVXdfaiq3pjkpiRbkzyV5Oe7+/7pMf86yTlJ/t10msu6+7ETzbXISQIAADDHCmBVbU5ya5LlJBcmubqqLlxTdl2SJ7r7/CQfSHLzdPxrSd7U3T+S5Jok96w57m3dfdH08dg6cwEAALCAeS4BvSTJwe5+pLufSnJvkl1ranYluWu6vTfJpVW1qbsf6u5Hp+MPJ/mu6QrfcznuXHP0CQAAwHOYJwBWksMz+0emY8et6e6jSZ5Msm1NzVuSPNTd35oZ++Wq+lxV3TgT8uaZK1V1fVUdqKoDKysrc5wGAADA2OYJgMdbfVt9PjVV9ZpMLuW8Yeb5t00vDf1b08fPPI/fl+6+rbt3dvfObdu+Ix8CAACwxjwB8EiSHTP725M8eqKaqtqS5Kwkj0/3tyf5eJK3d/eXjh3Q3T39+fUk/zKTS02fcy4AAABO3jwB8IEkF1TVeVW1NcnuJPvW1OzL5CYvSXJlkvu7e7Wqzk7ym0ne292/d6y4qrZU1fdPt89I8lNJ/uC55nr+pwYAAMCsdb8GoruPVtWeJPdl8jUQd3b3w1X1viQHuntfkjuS3FNVBzNZrds9PXxPkvOT3FhVN07HLkvyzST3TcPf5iSfTPLh6fMnmgsAAIAFbFpdffkvri0vL6/u379/o9sAAADYEFX1YHfvXK9unktAAQAAOA0IgAAAAIMQAAEAAAYhAAIAAAxCAAQAABiEAAgAADAIARAAAGAQAiAAAMAgBEAAAIBBCIAAAACDEAABAAAGIQACAAAMQgAEAAAYhAAIAAAwCAEQAABgEAIgAADAIARAAACAQQiAAAAAgxAAAQAABiEAAgAADEIABAAAGIQACAAAMAgBEAAAYBACIAAAwCAEQAAAgEEIgAAAAIMQAAEAAAYhAAIAAAxCAAQAABiEAAgAADAIARAAAGAQAiAAAMAgBEAAAIBBCIAAAACDEAABAAAGsWWeoqq6PMktSTYnub27b1rz/JlJ7k5ycZKVJFd196GqemOSm5JsTfJUkp/v7vur6hVJfi3JDyX5yySf6O73TOe6NskvJOnp9B/s7tsXOksAAADWXwGsqs1Jbk2ynOTCJFdX1YVryq5L8kR3n5/kA0luno5/LcmbuvtHklyT5J6ZY36xu384yY8l+YmqWp557mPdfdH0IfwBAAC8AOa5BPSSJAe7+5HufirJvUl2ranZleSu6fbeJJdW1abufqi7H52OP5zku6rqzO7+8+7+7SSZzvnZJNsXPRkAAABObJ4AWEkOz+wfmY4dt6a7jyZ5Msm2NTVvSfJQd3/rWQdWnZ3kTUk+NVtbVZ+vqr1VtWOOHgEAAFjHPAFw03HGVp9PTVW9JpPLQm+YLaiqLUk+muSfdfcj0+FPJDm3u380ySfzzMpi1hx7fVUdqKoDKysrc5wGAADA2OYJgEeSzK7CbU/y6IlqpqHurCSPT/e3J/l4krd395fWHHdbkj/u7l86NtDdKzOrhB/O5MYy36G7b+vund29c9u2tYuNAAAArDVPAHwgyQVVdV5VbU2yO8m+NTX7MrnJS5JcmeT+7l6dXt75m0ne292/N3tAVf3TTILiu9aMnzOze0WSL857MgAAAJzYul8D0d1Hq2pPkvsy+RqIO7v74ap6X5ID3b0vyR1J7qmqg5ms/O2eHr4nyflJbqyqG6djl2XytRD/JMkfJvlsVSXPfN3DO6vqiiRHp3Nd+4KcKQAAwOA2ra6u/Tjfy8/y8vLq/v37N7oNAACADVFVD3b3zvXq5rkEFAAAgNOAAAgAADAIARAAAGAQAiAAAMAgBEAAAIBBCIAAAACDEAABAAAGIQACAAAMQgAEAAAYhAAIAAAwCAEQAABgEAIgAADAIARAAACAQQiAAAAAgxAAAQAABiEAAgAADEIABAAAGIQACAAAMAgBEAAAYBACIAAAwCAEQAAAgEEIgAAAAIMQAAEAAAYhAAIAAAxCAAQAABiEAAgAADAIARAAAGAQAiAAAMAgBEAAAIBBCIAAAACDEAABAAAGIQACAAAMQgAEAAAYhAA4osOHk3e8I7nkksnPw4c3uqNn099i9LcY/S3uVO9Rf4vR32L0txj9LUZ/iznV+5vX6urqy/5x+eWXrzKnL395dfVVr1pdPeOM1dVk8vNVr5qMnwr0txj9LUZ/izvVe9TfYvS3GP0tRn+L0d9iTvX+VldXl5aWDqzOkZ2sAI7m/e9PvvGN5OmnJ/tPPz3Zf//7N7avY/S3GP0tRn+LO9V71N9i9LcY/S1Gf4vR32JO9f6ehy3zFFXV5UluSbI5ye3dfdOa589McneSi5OsJLmquw9V1RuT3JRka5Knkvx8d98/PebiJB9J8t1JfivJz3b3alV9X5KPJTk3yaEkb+3uJxY7Tb7t059+5oV7zNNPJ5/5zMb0s5b+FqO/xehvcad6j/pbjP4Wo7/F6G8x+lvMqd7f87DuCmBVbU5ya5LlJBcmubqqLlxTdl2SJ7r7/CQfSHLzdPxrSd7U3T+S5Jok98wc86Ek1ye5YPq4fDr+niSf6u4Lknxqus8L5cd/PDnjjGePnXHG5FrmU4H+FqO/xehvcad6j/pbjP4Wo7/F6G8x+lvMqd7f87HeNaJLS0t/c2lp6b6Z/fcuLS29d03NfUtLS39zur1laWnpa0tLS5vW1GxaWlpaWVpaOnNpaemcpaWlP5x57uqlpaX/dbr9R0tLS+dMt89ZWlr6o/V69BnA5+FUv35Zf4vR32L0t7hTvUf9LUZ/i9HfYvS3GP0t5lTvb/WF/QxgJZm9xc2R6dhxa7r7aJInk2xbU/OWJA9197em9UdOMOcPdPdXpnN9Jcmrj9tU1fVVdaCqDqysrMxxGiRJduxIfv/3kxtumPzF4oYbJvs7dmx0ZxP6W4z+FqO/xZ3qPepvMfpbjP4Wo7/F6G8xp3p/z8Om1dXV5yyoqp9O8p919z+Y7v9Mkku6+x0zNQ9Pa45M9780rVmZ7r8myb4kl3X3l6rq9Un+p+7+u9Pn/1aSd3f3m6rqz7r77Jm5n+juVz1Xj8vLy6v79+9/3icPAABwOqiqB7t753p186wAHkkyG223J3n0RDVVtSXJWUken+5vT/LxJG/v7i/N1G8/wZxfrapzpseek+SxOXoEAABgHfMEwAeSXFBV51XV1iS7M1nNm7Uvk5u8JMmVSe6f3tHz7CS/meS93f17x4qnl3Z+vareUFWbkrw9yW8cZ65rZsYBAABYwLoBcPqZvj1J7kvyxSS/2t0PV9X7quqKadkdSbZV1cEk/02euXPnniTnJ7mxqj43fRz7TN8/THJ7koNJvpTk2DWcNyV5Y1X9cZJjXyMBAADAgtb9DODLgc8AAgAAI3shPwMIAADAaUAABAAAGIQACAAAMAgBEAAAYBACIAAAwCAEQAAAgEEIgAAAAIMQAAEAAAYhAAIAAAxCAAQAABiEAAgAADAIARAAAGAQAiAAAMAgBEAAAIBBCIAAAACDEAABAAAGIQACAAAMQgAEAAAYhAAIAAAwCAEQAABgEAIgAADAIARAAACAQQiAAAAAgxAAAQAABiEAAgAADEIABAAAGIQACAAAMAgBEAAAYBACIAAAwCAEQAAAgEEIgAAAAIMQAAEAAAYhAAIAAAxCAAQAABiEAAgAADCILfMUVdXlSW5JsjnJ7d1905rnz0xyd5KLk6wkuaq7D1XVtiR7k7w+yUe6e8+0/nuT/M7MFNuT/IvufldVXZvkF5L09LkPdvftJ3l+AAAATK0bAKtqc5Jbk7wxyZEkD1TVvu7+wkzZdUme6O7zq2p3kpuTXJXkL5LcmOS100eSpLu/nuSimd/xYJJfn5nvY8fCIgAAAC+MeS4BvSTJwe5+pLufSnJvkl1ranYluWu6vTfJpVW1qbu/2d2/m0kQPK6quiDJq/PsFUEAAABeYPMEwEpyeGb/yHTsuDXdfTTJk0m2zdnD1Zms+K3OjL2lqj5fVXurasec8wAAAPAc5gmAm44ztnoSNSeyO8lHZ/Y/keTc7v7RJJ/MMyuLz1JV11fVgao6sLKyMuevAgAAGNc8AfBIktlVuO1JHj1RTVVtSXJWksfXm7iqXpdkS3c/eGysu1e6+1vT3Q9ncmOZ79Ddt3X3zu7euW3bvIuNAAAA45onAD6Q5IKqOq+qtmayYrdvTc2+JNdMt69Mcv+aSzpP5Oo8e/UvVXXOzO4VSb44xzwAAACsY927gHb30arak+S+TL4G4s7ufriq3pfkQHfvS3JHknuq6mAmK3+7jx1fVYeSvDLJ1qp6c5LLZu4g+tYkP7nmV76zqq5IcnQ617ULnB8AAABTm1ZX5/2o3qlreXl5df/+/RvdBgAAwIaoqge7e+d6dfNcAgoAAMBpQAAEAAAYhAAIAAAwCAEQAABgEAIgAADAIARAAACAQQiAAAAAgxAAAQAABiEAAgAADEIABAAAGIQACAAAMAgBEAAAYBACIAAAwCAEQAAAgEEIgAAAAIMQAAEAAAYhAAIAAAxCAAQAABiEAAgAADAIARAAAGAQAiAAAMAgBEAAAIBBCIAAAACDEAABAAAGIQACAAAMQgAEAAAYhAAIAAAwCAEQAABgEAIgAADAIARAAACAQQiAAAAAgxAAAQAABiEAAgAADEIAfDEcPpy84x3JJZdMfh4+vNEdAQAAZMtGN3DaOXw4ed3rkm98I3n66eRzn0t+5VeS3//9ZMeOje4OAAAYmBXAF9r73/9M+EsmP7/xjck4AADABpprBbCqLk9yS5LNSW7v7pvWPH9mkruTXJxkJclV3X2oqrYl2Zvk9Uk+0t17Zo7510nOSfLvpkOXdfdjJ5rrpM/wpfbpTz8T/o55+unkM5/ZmH4AAACm1l0BrKrNSW5NspzkwiRXV9WFa8quS/JEd5+f5ANJbp6O/0WSG5P83Ammf1t3XzR9PLbOXC8PP/7jyRlnPHvsjDMmnwcEAADYQPNcAnpJkoPd/Uh3P5Xk3iS71tTsSnLXdHtvkkuralN3f7O7fzeTIDiv4871PI7fWO9+d/I93/NMCDzjjMn+u9+9sX0BAADDmycAVpLZ21gemY4dt6a7jyZ5Msm2Oeb+5ar6XFXdOBPy5pqrqq6vqgNVdWBlZWWOX/US2bFjcsOXG26YrPrdcIMbwAAAAKeEeT4DeLzVt9WTqFnrbd3dVfW9Sf5Vkp/J5LN/c83V3bcluS1JlpeX1/tdL60dO5J//s83ugsAAIBnmWcF8EiS2eWr7UkePVFNVW1JclaSx59r0u7u6c+vJ/mXmVxqelJzAQAAsL55AuADSS6oqvOqamuS3Un2ranZl+Sa6faVSe7v7hOuylXVlqr6/un2GUl+KskfnMxcAAAAzGfdS0C7+2hV7UlyXyZfA3Fndz9cVe9LcqC79yW5I8k9VXUwk9W63ceOr6pDSV6ZZGtVvTnJZUn+JMl90/C3Ocknk3x4esgJ5wIAAODkbVpdffkvri0vL6/u379/o9sAAADYEFX1YHfvXK9unktAAQAAOA0IgAAAAIMQAAEAAAYhAAIAAAxCAAQAABiEAAgAADAIARAAAGAQAiAAAMAgBEAAAIBBCIAAAACDEAABAAAGIQACAAAMQgAEAAAYhAAIAAAwCAEQAABgEAIgAADAIARAAACAQQiAAAAAgxAAAQAABiEAAgAADEIABAAAGIQACAAAMAgBEAAAYBACIAAAwCAEQAAAgEEIgAAAAIMQAAEAAAYhAAIAAAxCAAQAABiEAAgAADAIARAAAGAQAiAAAMAgBEAAAIBBCIAAAACDEAABAAAGsWWeoqq6PMktSTYnub27b1rz/JlJ7k5ycZKVJFd196Gq2pZkb5LXJ/lId++Z1r8iya8l+aEkf5nkE939nulz1yb5hSQ9nf6D3X37IicJAADAHCuAVbU5ya1JlpNcmOTqqrpwTdl1SZ7o7vOTfCDJzdPxv0hyY5KfO87Uv9jdP5zkx5L8RFUtzzz3se6+aPoQ/gAAAF4A81wCekmSg939SHc/leTeJLvW1OxKctd0e2+SS6tqU3d/s7t/N5Mg+G3d/efd/dvT7aeSfDbJ9gXOAwAAgHXMEwAryeGZ/SPTsePWdPfRJE8m2TZPA1V1dpI3JfnUzPBbqurzVbW3qnbMMw8AAADPbZ4AuOk4Y6snUfMdqmpLko8m+Wfd/ch0+BNJzu3uH03yyTyzsrj22Our6kBVHVhZWVnvVwEAAAxvngB4JMnsKtz2JI+eqGYa6s5K8vgcc9+W5I+7+5eODXT3Snd/a7r74UxuLPMduvu27t7Z3Tu3bZtrsREAAGBo8wTAB5JcUFXnVdXWJLuT7FtTsy/JNdPtK5Pc393PuQJYVf80k6D4rjXj58zsXpHki3P0CAAAwDrW/RqI7j5aVXuS3JfJ10Dc2d0PV9X7khzo7n1J7khyT1UdzGTlb/ex46vqUJJXJtlaVW9OclmSf5vknyT5wySfrarkma97eGdVXZHk6HSua1+gcwUAABjaptXVdT+qd8pbXl5e3b9//0a3AQAAsCGq6sHu3rle3TyXgAIAAHAaEAABAAAGIQACAAAMQgAEAAAYhAAIAAAwCAEQAABgEAIgAADAIARAAACAQQiAAAAAgxAAAQAABiEAAgAADEIABAAAGIQACAAAMAgBEAAAYBACIAAAwCAEQAAAgEEIgAAAAIMQAAEAAAYhAAIAAAxCAAQAABiEAAgAADAIARAAAGAQAiAAAMAgBEAAAIBBCIAAAACDEAABAAAGIQACAAAMQgAEAAAYhAAIAAAwCAEQAABgEAIgAADAIARAAACAQQiAAAAAgxAAAQAABiEAAgAADEIABAAAGMSWeYqq6vIktyTZnOT27r5pzfNnJrk7ycVJVpJc1d2Hqmpbkr1JXp/kI929Z+aYi5N8JMl3J/mtJD/b3atV9X1JPpbk3CSHkry1u59Y4BwBAADIHCuAVbU5ya1JlpNcmOTqqrpwTdl1SZ7o7vOTfCDJzdPxv0hyY5KfO87UH0pyfZILpo/Lp+PvSfKp7r4gyaem+wAAACxonktAL0lysLsf6e6nktybZNeaml1J7ppu701yaVVt6u5vdvfvZhIEv62qzknyyu7+v7p7NZPVwzcfZ667ZsYBAABYwDwBsJIcntk/Mh07bk13H03yZJJt68x55ARz/kB3f2U611eSvPq4E1RdX1UHqurAysrKHKcBAAAwtnk+A7jpOGOrJ1GzSP136O7bktyWJMvLy8/rWAAAgBHNswJ4JMmOmf3tSR49UU1VbUlyVpLH15lz+wnm/Or0EtFjl4o+NkePAAAArGOeAPhAkguq6ryq2ppkd5J9a2r2Jblmun1lkvunn+07rumlnV+vqjdU1aYkb0/yG8eZ65qZcQAAABaw7iWg3X20qvYkuS+Tr4G4s7sfrqr3JTnQ3fuS3JHknqo6mMnK3+5jx1fVoSSvTLK1qt6c5LLu/kKSf5hnvgZi//SRJDcl+dWqui7Jl5P89AtxogAAAKPbtLr68v/43PLy8ur+/fvXLwQAADgNVdWD3b1zvbp5LgEFAADgNCAAAgAADEIABAAAGIQACAAAMAgBEAAAYBACIAAAwCAEQAAAgEEIgAAAAIMQAAEAAAYhAAIAAAxCAAQAABiEAAgAADAIARAAAGAQAiAAAMAgBEAAAIBBCIAAAACDEAABAAAGIQACAAAMQgAEAAAYhAAIAAAwCAEQAABgEAIgAADAIARAAACAQQiAAAAAgxAAAQAABiEAAgAADEIABAAAGIQACAAAMAgBEAAAYBACIAAAwCAEQAAAgEEIgAAAAIMQAAEAAAYhAAIAAAxCAAQAABjElnmKquryJLck2Zzk9u6+ac3zZya5O8nFSVaSXNXdh6bPvTfJdUn+Msk7u/u+qvobST42M8UPJvnvuvuXqup/SPJfJfnT6XP/uLt/6+RODwAAgGPWDYBVtTnJrUnemORIkgeqal93f2Gm7LokT3T3+VW1O8nNSa6qqguT7E7ymiRLST5ZVX+9u/8oyUUz83eSj8/M94Hu/sXFTw8AAIBj5rkE9JIkB7v7ke5+Ksm9SXatqdmV5K7p9t4kl1bVpun4vd39re7+N0kOTuebdWmSL3X3n5zsSQAAALC+eQJgJTk8s39kOnbcmu4+muTJJNvmPHZ3ko+uGdtTVZ+vqjur6lVz9AgAAMA65gmAm44ztjpnzXMeW1Vbk1yR5Ndmnv9Qkh/K5BLRryT5n4/XVFVdX1UHqurAysrKibsHAAAgyXw3gTmSZMfM/vYkj56g5khVbUlyVpLH5zh2Oclnu/urxwZmt6vqw0n+9+M11d23JbktSZaXl9cGUgAAANaYZwXwgSQXVNV50xW73Un2ranZl+Sa6faVSe7v7tXp+O6qOrOqzktyQZLPzBx3ddZc/llV58zs/r0kfzDvyQAAAHBi664AdvfRqtqT5L5Mvgbizu5+uKrel+RAd+9LckeSe6rqYCYrf7unxz5cVb+a5AtJjib5R939l0lSVa/I5M6iN6z5le+vqosyuVT00HGeBwAA4CRsWl19+V89uby8vLp///6NbgMAAGBDVNWD3b1zvbp5LgEFAADgNCAAAgAADEIABAAAGIQACAAAMIjT4iYwVfWnSf5ko/tgON+f5Gsb3QTD8vpjo3kNspG8/thIp+rr7z/o7r+2XtFpEQBhI1TVgXnutAQvBq8/NprXIBvJ64+N9HJ//bkEFAAAYBACIAAAwCAEQDh5t210AwzN64+N5jXIRvL6YyO9rF9/PgMIAAAwCCuAAAAAg9iy0Q3Ay0FV7Uhyd5J/L8n/l+S27r6lqr4vyceSnJvkUJK3dvcTG9Unp7eq2pzkQJLu7p+qqvOS3Jvk+5J8NsnPdPdTG9kjp6eqOjvJ7Ulem2Q1yX+Z5I/i/Y+XQFX910n+QSavvf8nyd9Pck68//Eiqao7k/xUkse6+7XTseP+n6+qNiW5JclPJvnzJNd292c3ou95WQGE+RxN8t9293+Y5A1J/lFVXZjkPUk+1d0XJPnUdB9eLD+b5Isz+zcn+cD09fdEkus2pCtGcEuS/6O7fzjJ6zJ5HXr/40VXVZXknUl2Tv8jvjnJ7nj/48X1kSSXrxk70XvecpILpo/rk3zoJerxpAmAMIfu/sqxv+Z099cz+c9PJdmV5K5p2V1J3rwxHXK6q6rtSf7zTFZhMv2L499Jsnda4vXHi6KqXpnkP01yR5J091Pd/Wfx/sdLZ0uS766qLUlekeQr8f7Hi6i7/88kj68ZPtF73q4kd3f3anf/30nOrqpzXppOT44ACM9TVZ2b5MeSfDrJD3T3V5JJSEzy6g1sjdPbLyV5dyaXICfJtiR/1t1Hp/tHMvmjBLzQfjDJnyb55ap6qKpur6q/Gu9/vAS6u5P8YpIvZxL8nkzyYLz/8dI70XvM7J1uAAACAklEQVReJTk8U3fKvx4FQHgequp7kvyrJO/q7n+70f0whqo69jmEB2eGNx2n1G2deTFsSfIfJflQd/9Ykm/G5Z68RKrqVZmssJyXZCnJX83kkru1vP+xUV52/x4LgDCnqjojk/D3K93969Phrx5b5p/+fGyj+uO09hNJrqiqQ5nc9ODvZLIiePb0kqgk2Z7k0Y1pj9PckSRHuvvT0/29mQRC73+8FP5ukn/T3X/a3U8n+fUk/3G8//HSO9F73pEkO2bqTvnXowAIc5h+3uqOJF/s7v9l5ql9Sa6Zbl+T5Dde6t44/XX3e7t7e3efm8nND+7v7rcl+e0kV07LvP54UXT3/5vkcFX9jenQpUm+EO9/vDS+nOQNVfWK6b/Fx15/3v94qZ3oPW9fkrdX1aaqekOSJ49dKnqq8kXwMIeq+k+S/E4mt58+9hmsf5zJ5wB/Ncm/n8k/Uj/d3Ws/NAwvmKr620l+bvo1ED+YZ26D/lCS/6K7v7WR/XF6qqqLMrkB0dYkj2RyG/6/Eu9/vASq6n9MclUmd+R+KJOvhKh4/+NFUlUfTfK3k3x/kq8m+e+T/G85znve9A8TH8zkrqF/nuTvd/eBjeh7XgIgAADAIFwCCgAAMAgBEAAAYBACIAAAwCAEQAAAgEEIgAAAAIMQAAEAAAYhAAIAAAxCAAQAABjE/w9lJmgVI+LMwQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x112af1710>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(15,7))\n",
    "plt.scatter(x=ntree[1:nsimu],y=accuracy[1:nsimu],s=30,c='red')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 178,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "nsimu = 21\n",
    "accuracy = [0]*nsimu\n",
    "ntree = [0]*nsimu\n",
    "for i in range(1,nsimu):\n",
    "    rf = RandomForestClassifier(n_estimators=i*5,min_samples_split=10,max_depth=None,criterion='entropy')\n",
    "    rf.fit(X_train,y_train)\n",
    "    rf_pred = rf.predict(X_test)\n",
    "    cm = confusion_matrix(y_test,rf_pred)\n",
    "    accuracy[i] = (cm[0,0]+cm[1,1])/cm.sum()\n",
    "    ntree[i]=i*5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 179,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA3oAAAGfCAYAAAAJRaBgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAHfZJREFUeJzt3X+w5XV93/HX7S5gY0Ywm5jmvndbsWDtSlqVDZqk7dgQHG4bXSfSsqSJJCGDtdn0h9FG4jA1mD/AMTVkdOwwYAWmE6IkNJs4W/4I+VEdqyySaNFmslCUfWOjLIRGnQhbb/84Z/Vyvcs97IG9u599PGZ29ny/53POfX9nzhx47vd7zl1YXl4OAAAA4/hrGz0AAAAATy+hBwAAMBihBwAAMBihBwAAMBihBwAAMBihBwAAMBihBwAAMBihBwAAMBihBwAAMJjNsyyqqguTXJtkU5Lru/vqVfefluSmJOcmOZjk4u6+v6ouSHJ1klOTPJbkLd19x/QxlyT5xSTLSR5M8uPd/dCTzXHOOecsb9u27SkcHgAAwDg+9alPPdTd37XeunVDr6o2JXlvkguSHEhyZ1Xt6e7PrFh2WZJHuvusqtqV5JokFyd5KMmru/vBqjonye2Tp6zNmYTj9u5+qKremWR3krc/2Szbtm3L3r171xsZAABgSFX1uVnWzXLp5nlJ9nf3fd39WJJbkuxctWZnkhunt29Ncn5VLXT33d394HT/PUmeNT37tzD98+yqWkjynEzO6gEAADCnWS7drCQPrNg+kOTlR1rT3Yeq6tEkWzI5o3fY65Lc3d1fS5KqemOSTyf5SpI/S/Kza/7wqsuTXD69PcO4AAAAJ7dZzugtrLFv+amsqaoXZ3I55xum26ckeWOSlyZZTPKpJFes9cO7+7ru3tHdO7Zs2TLDuAAAACe3WULvQJKV34CyNd96meU31kw/f3d6koen21uT3Jbk9d1973T9S5Kku+/t7uUkH0zyA0d5DAAAAKwwS+jdmeTsqjqzqk5NsivJnlVr9iS5dHr7oiR3dPdyVZ2R5MNJrujuj65Y30m2V9Xhb4u5IMlnj/YgAAAA+KZ1P6M3/czd7ky+MXNTkvd39z1VdVWSfd29J8kNSW6uqv2ZnMnbNX347iRnJbmyqq6c7nvV9Fs4fynJH1XV40k+l+Qnn84DAwAAOFktLC+v/rjd8WtpaWnZr1cAAABOVlV1V3fvWG/dLJduAgAAcAIRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIPZPMuiqrowybVJNiW5vruvXnX/aUluSnJukoNJLu7u+6vqgiRXJzk1yWNJ3tLdd0wfc2qS9yR5ZZKvJ3lbd//m03FQAAAAJ7N1z+hV1aYk702ylGR7kkuqavuqZZcleaS7z0ry7iTXTPc/lOTV3f29SS5NcvOKx7wtyRe7+4XT5/3DeQ4EAACAiVnO6J2XZH9335ckVXVLkp1JPrNizc4kb5/evjXJe6pqobvvXrHmniTPqqrTuvtrSX46yYuSpLu/nkkUAgAAMKdZQq+SPLBi+0CSlx9pTXcfqqpHk2zJE+PtdUnu7u6vVdUZ033vqKpXJrk3ye7u/vNv+eFVlye5fHp7hnEBAABObrN8GcvCGvuWn8qaqnpxJpdzvmG6a3OSrUk+2t0vS/KxJO9a64d393XdvaO7d2zZsmWGcQEAAE5us4TegSTbVmxvTfLgkdZU1eYkpyd5eLq9NcltSV7f3fdO1x9M8tXp/iT5UJKXHcX8AAAArDJL6N2Z5OyqOnP6TZm7kuxZtWZPJl+2kiQXJbmju5enl2h+OMkV3f3Rw4u7eznJ72TyjZtJcn6e+Jk/AAAAjtK6odfdh5LsTnJ7ks8m+WB331NVV1XVa6bLbkiypar2J3lTkrdO9+9OclaSK6vqj6d/nje97xeSvL2qPpXkJ5L8/NN2VAAAACexheXl1R+3O34tLS0t7927d6PHAAAA2BBVdVd371hv3SyXbgIAAHACEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACD2TzLoqq6MMm1STYlub67r151/2lJbkpybpKDSS7u7vur6oIkVyc5NcljSd7S3XeseuyeJC/o7nPmPRgAAABmOKNXVZuSvDfJUpLtSS6pqu2rll2W5JHuPivJu5NcM93/UJJXd/f3Jrk0yc2rnvtHk3x5riMAAADgCWa5dPO8JPu7+77ufizJLUl2rlqzM8mN09u3Jjm/qha6++7ufnC6/54kz5qe/UtVfXuSNyX55XkPAgAAgG+a5dLNSvLAiu0DSV5+pDXdfaiqHk2yJZMzeoe9Lsnd3f216fY7kvxKkq8+6Q+vujzJ5dPbM4wLAABwcpvljN7CGvuWn8qaqnpxJpdzvmG6/ZIkZ3X3bev98O6+rrt3dPeOLVu2zDAuAADAyW2W0DuQZNuK7a1JHjzSmqranOT0JA9Pt7cmuS3J67v73un6709yblXdn+QjSV5YVX9wdIcAAADASrNcunlnkrOr6swknWRXkh9btWZPJl+28rEkFyW5o7uXq+qMJB9OckV3f/Tw4u5+X5L3JUlVPT/J73b3K+c7FAAAAJIZzuh196Eku5PcnuSzST7Y3fdU1VVV9ZrpshuSbKmq/Zl8wcpbp/t3JzkryZVV9cfTP8972o8CAACAb1hYXl79cbvj19LS0vLevXs3egwAAIANUVV3dfeO9dbN8hk9AAAATiBCDwAAYDBCDwAAYDBCDwAAYDBCDwAAYDBCDwAAYDBCDwAAYDBCDwAAYDBCDwAAYDBCDwAAYDBCDwAAYDBCDwAAYDBCDwAAYDBCDwAAYDBCDwAAYDBCDwAAYDBCDwAAYDBCDwAAYDBCDwAAYDBCDwAAYDBCDwAAYDBCDwAAYDBCDwAAYDBCDwAAYDBCDwAAYDBCDwAAYDBCDwAAYDBCDwAAYDBCDwAAYDBCDwAAYDBCDwAAYDBCDwAAYDBCDwAAYDBCDwAAYDBCDwAAYDBCDwAAYDBCDwAAYDBCDwAAYDBCDwAAYDCbZ1lUVRcmuTbJpiTXd/fVq+4/LclNSc5NcjDJxd19f1VdkOTqJKcmeSzJW7r7jqr6tiQfSvK3k/y/JL/T3W99mo4JAADgpLbuGb2q2pTkvUmWkmxPcklVbV+17LIkj3T3WUneneSa6f6Hkry6u783yaVJbl7xmHd194uSvDTJD1bV0lxHAgAAQJLZLt08L8n+7r6vux9LckuSnavW7Exy4/T2rUnOr6qF7r67ux+c7r8nybOq6rTu/mp3/36STJ/zk0m2znswAAAAzBZ6leSBFdsHpvvWXNPdh5I8mmTLqjWvS3J3d3/tCQ+sOiPJq5P83po/vOryqtpXVfsOHjw4w7gAAAAnt1k+o7ewxr7lp7Kmql6cyeWcr1q5oKo2J/n1JL/W3fet9cO7+7ok1yXJ0tLS6p8LAADAKrOc0TuQZNuK7a1JHjzSmmm8nZ7k4en21iS3JXl9d9+76nHXJfmz7v7Vpz46AAAAa5nljN6dSc6uqjOTdJJdSX5s1Zo9mXzZyseSXJTkju5enl6W+eEkV3T3R1c+oKp+OZMg/Jn5DgEAAICV1g297j5UVbuT3J7Jr1d4f3ffU1VXJdnX3XuS3JDk5qran8mZvF3Th+9OclaSK6vqyum+V2Xy6xbeluR/JflkVSXJe7r7+qfv0AAAAE5OC8vLJ87H3paWlpb37t270WMAAABsiKq6q7t3rLduls/oAQAAcAIRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRevN44IHk534uOe+8yd8PPLDREz2R+eZjvvmYbz7mm4/55mO++ZhvPuab3/E+o/mOjeXl5RPmz4UXXrh83Pj855eXn/vc5eVTTlleTiZ/P/e5k/3HA/PNx3zzMd98zDcf883HfPMx33zMN7/jfUbzzW1xcXHf8gzt5Ize0XrnO5Mvfzl5/PHJ9uOPT7bf+c6Nnesw883HfPMx33zMNx/zzcd88zHffMw3v+N9RvMdM0LvaH384998ARz2+OPJJz6xMfOsZr75mG8+5puP+eZjvvmYbz7mm4/55ne8z2i+Y2am0KuqC6vqT6tqf1W9dY37T6uq35je//Gqev50/wVVdVdVfXr69w+teMy50/37q+rXqmrhaTuqY+HlL09OOeWJ+045ZXIt7/HAfPMx33zMNx/zzcd88zHffMw3H/PN73if0XzHznrXdi4uLm5aXFy8d3Fx8QWLi4unLi4u/sni4uL2VWv+1eLi4n+a3t61uLj4G9PbL11cXFyc3j5ncXGxVzzmE4uLi9+/uLi4sLi4uHdxcXFpvVl8Ru8pMN98zDcf883HfPMx33zMNx/zzcd88zveZzTf3J7Oz+idl2R/d9/X3Y8luSXJzlVrdia5cXr71iTnV9VCd9/d3Q9O99+T5FnTs3/fk+Q53f2x7l5OclOS185drcfStm3Jn/xJ8oY3TAr/DW+YbG/bttGTTZhvPuabj/nmY775mG8+5puP+eZjvvkd7zOa75hZWF5eftIFVXVRkgu7+2em2z+R5OXdvXvFmv85XXNgun3vdM1Dq57nX3b3D1fVjiRXd/cPT+/7h0l+obt/ZI2ff3mSy6e3z/3ECXh9LAAAwNOhqu7q7h3rrds8w3Ot9dm51XX4pGuq6sVJrknyqqfwnEmS7r4uyXVJsrS09ORVCgAAwEyXbh5IsvJc5dYkDx5pTVVtTnJ6koen21uT3Jbk9d1974r1W9d5TgAAAI7CLKF3Z5Kzq+rMqjo1ya4ke1at2ZPk0unti5Lc0d3LVXVGkg8nuaK7P3p4cXd/IclfVtUrpt+2+fokvz3nsQAAAJAZQq+7DyXZneT2JJ9N8sHuvqeqrqqq10yX3ZBkS1XtT/KmJId/BcPuJGclubKq/nj653nT+96Y5Pok+5Pcm2Tv03VQAAAAJ7N1v4zleLK0tLS8d68eBAAATk6zfhnLTL8wHQAAgBOH0AMAABiM0AMAABiM0AMAABiM0AMAABiM0AMAABiM0AMAABiM0AMAABiM0AMAABiM0AMAABiM0AMAABiM0AMAABiM0AMAABiM0AMAABiM0AMAABiM0AMAABiM0AMAABiM0AMAABiM0AMAABiM0AMAABiM0AMAABiM0AMAABiM0AMAABiM0AMAABiM0AMAABiM0AMAABiM0AMAABiM0AMAABiM0AMAABiM0AMAABiM0AMAABiM0AMAABiM0AMAABiM0AMAABiM0AMAABiM0AMAABiM0AMAABiM0AMAABiM0AMAABjM5lkWVdWFSa5NsinJ9d199ar7T0tyU5JzkxxMcnF3319VW5LcmuT7knygu3eveMwlSX4xyXKSB5P8eHc/NP8hAQAAnNzWPaNXVZuSvDfJUpLtSS6pqu2rll2W5JHuPivJu5NcM93/V0muTPLmVc+5OZNw/Mfd/feSfCrJ7gAAADC3WS7dPC/J/u6+r7sfS3JLkp2r1uxMcuP09q1Jzq+qhe7+Snd/JJPgW2lh+ufZVbWQ5DmZnNUDAABgTrOEXiV5YMX2gem+Ndd096EkjybZcqQn7O7Hk7wxyaczCbztSW5Y84dXXV5V+6pq38GDB2cYFwAA4OQ2S+gtrLFv+SjWfENVnZJJ6L00yWIml25esdba7r6uu3d0944tW47YjgAAAEzNEnoHkmxbsb0133qZ5TfWTD9/d3qSh5/kOV+SJN19b3cvJ/lgkh+YcWYAAACexCyhd2eSs6vqzKo6NcmuJHtWrdmT5NLp7YuS3DENuCPpJNur6rum2xck+ezsYwMAAHAk6/56he4+VFW7k9yeya9XeH9331NVVyXZ1917Mvl83c1VtT+TM3m7Dj++qu7P5MtWTq2q1yZ5VXd/pqp+KckfVdXjST6X5Cef3kMDAAA4OS0sLz/Zibfjy9LS0vLevXs3egwAAIANUVV3dfeO9dbNcukmAAAAJxChBwAAMBihBwAAMBihBwAAMBihBwAAMBihBwAAMBihBwAAMBihBwAAMBihBwAAMBihBwAAMBihBwAAMBihBwAAMBihBwAAMBihBwAAMBihBwAAMBihBwAAMBihBwAAMBihBwAAMBihBwAAMBihBwAAMBihBwAAMBihBwAAMBihBwAAMBihBwAAMBihBwAAMBihBwAAMBihBwAAMBihBwAAMBihBwAAMBihBwAAMBihBwAAMBihBwAAMBihBwAAMBihBwAAMBihBwAAMBihBwAAMBihBwAAMBihBwAAMBihBwAAMJjNsyyqqguTXJtkU5Lru/vqVfefluSmJOcmOZjk4u6+v6q2JLk1yfcl+UB3717xmFOTvCfJK5N8Pcnbuvs35z4iAACAk9y6Z/SqalOS9yZZSrI9ySVVtX3VssuSPNLdZyV5d5Jrpvv/KsmVSd68xlO/LckXu/uF0+f9w6M6AgAAAJ5gljN65yXZ3933JUlV3ZJkZ5LPrFizM8nbp7dvTfKeqlro7q8k+UhVnbXG8/50khclSXd/PclDR3UEAAAAPMEsn9GrJA+s2D4w3bfmmu4+lOTRJFuO+IRVZ0xvvqOqPllVH6qq7555agAAAI5oltBbWGPf8lGsWWlzkq1JPtrdL0vysSTvWmthVV1eVfuqat/BgwdnGBcAAODkNkvoHUiybcX21iQPHmlNVW1OcnqSh5/kOQ8m+WqS26bbH0rysrUWdvd13b2ju3ds2XLEk4QAAABMzRJ6dyY5u6rOnH5T5q4ke1at2ZPk0unti5Lc0d1HPKM3ve93MvnGzSQ5P0/8zB8AAABHad0vY+nuQ1W1O8ntmfx6hfd39z1VdVWSfd29J8kNSW6uqv2ZnMnbdfjxVXV/kuckObWqXpvkVd39mSS/MH3Mryb5UpKfenoPDQAA4OS0sLz8ZB+lO74sLS0t7927d6PHAAAA2BBVdVd371hv3SyXbgIAAHACEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACDEXoAAACD2TzLoqq6MMm1STYlub67r151/2lJbkpybpKDSS7u7vurakuSW5N8X5IPdPfuNZ57T5IXdPc5cx0JAAAASWY4o1dVm5K8N8lSku1JLqmq7auWXZbkke4+K8m7k1wz3f9XSa5M8uYjPPePJvny0Y0OAADAWma5dPO8JPu7+77ufizJLUl2rlqzM8mN09u3Jjm/qha6+yvd/ZFMgu8Jqurbk7wpyS8f9fQAAAB8i1lCr5I8sGL7wHTfmmu6+1CSR5NsWed535HkV5J8daZJAQAAmMksobewxr7lo1jzDVX1kiRndfdt6/3wqrq8qvZV1b6DBw+utxwAAOCkN0voHUiybcX21iQPHmlNVW1OcnqSh5/kOb8/yblVdX+SjyR5YVX9wVoLu/u67t7R3Tu2bFnvJCEAAACzfOvmnUnOrqozk3SSXUl+bNWaPUkuTfKxJBcluaO7j3hGr7vfl+R9SVJVz0/yu939yqc6PAAAAN9q3dDr7kNVtTvJ7Zn8eoX3d/c9VXVVkn3dvSfJDUlurqr9mZzJ23X48dOzds9JcmpVvTbJq7r7M0//oQAAAJAkC8vLRzzxdtxZWlpa3rt370aPAQAAsCGq6q7u3rHeulk+owcAAMAJROgBAAAMRugBAAAMRugBAAAMRugBAAAMRugBAAAMRugBAAAMRugBAAAMRugBAAAMRugBAAAMRugBAAAMRugBAAAMRugBAAAMRugBAAAMRugBAAAMRugBAAAMRugBAAAMRugBAAAMRugBAAAMRugBAAAMRugBAAAMRugBAAAMRugBAAAMRugBAAAMRugBAAAMRugBAAAMRugBAAAMRugBAAAMRugBAAAMRugBAAAMRugBAAAMRugBAAAMRugBAAAMRugBAAAMRugBAAAMRugBAAAMZmF5eXmjZ5hZVX0pyec2eg5OOt+Z5KGNHoKTltcfG8nrj43k9cdGO15fg3+ru79rvUUnVOjBRqiqfd29Y6Pn4OTk9cdG8vpjI3n9sdFO9NegSzcBAAAGI/QAAAAGI/Rgfddt9ACc1Lz+2Ehef2wkrz822gn9GvQZPQAAgME4owcAADCYzRs9ABwvqmpbkpuS/I0kX09yXXdfW1XfkeQ3kjw/yf1J/nl3P7JRczK2qtqUZF+S7u4fqaozk9yS5DuSfDLJT3T3Yxs5I+OqqjOSXJ/knCTLSX46yZ/GeyDHQFX9uyQ/k8lr79NJfirJ98R7IM+Aqnp/kh9J8sXuPme6b83/56uqhSTXJvknSb6a5Ce7+5MbMfdT4YwefNOhJD/f3X83ySuS/GxVbU/y1iS/191nJ/m96TY8U/5Nks+u2L4mybunr79Hkly2IVNxsrg2yX/r7hcl+fuZvBa9B/KMq6pK8q+T7Jj+T/emJLviPZBnzgeSXLhq35He75aSnD39c3mS9x2jGeci9GCqu79w+F9nuvsvM/kfnEqyM8mN02U3JnntxkzI6Kpqa5J/mskZlUz/BfGHktw6XeL1xzOmqp6T5B8luSFJuvux7v6LeA/k2Nmc5K9X1eYk35bkC/EeyDOku/8oycOrdh/p/W5nkpu6e7m7/0eSM6rqe47NpEdP6MEaqur5SV6a5ONJvru7v5BMYjDJ8zZwNMb2q0n+fSaXDifJliR/0d2HptsHMvnHB3gmvCDJl5L856q6u6qur6pnx3sgx0B3d5J3Jfl8JoH3aJK74j2QY+tI73eV5IEV606I16LQg1Wq6tuT/GaSf9vd/3ej5+HkUFWHPydw14rdC2ss9VXJPFM2J3lZkvd190uTfCUu0+QYqarnZnLW5Mwki0mencnlcqt5D2QjnJD/PRZ6sEJVnZJJ5P2X7v6t6e4/P3x6fvr3FzdqPob2g0leU1X3Z/LFAz+UyRm+M6aXMSXJ1iQPbsx4nAQOJDnQ3R+fbt+aSfh5D+RY+OEk/7u7v9Tdjyf5rSQ/EO+BHFtHer87kGTbinUnxGtR6MHU9PNQNyT5bHf/xxV37Uly6fT2pUl++1jPxvi6+4ru3trdz8/kCwju6O5/keT3k1w0Xeb1xzOmu/9Pkgeq6u9Md52f5DPxHsix8fkkr6iqb5v+9/jw6897IMfSkd7v9iR5fVUtVNUrkjx6+BLP45lfmA5TVfUPkvz3TL7S+fBnpH4xk8/pfTDJ38zkP0T/rLtXf3gXnjZV9cokb57+eoUX5JtfLX53kh/v7q9t5HyMq6peksmXAZ2a5L5Mvt7+r8V7IMdAVf1Skosz+RbsuzP5VQsV74E8A6rq15O8Msl3JvnzJP8hyX/NGu930398eE8m39L51SQ/1d37NmLup0LoAQAADMalmwAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIMRegAAAIP5/xvYbGwIrj+QAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x112af15c0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(15,7))\n",
    "plt.scatter(x=ntree[1:nsimu],y=accuracy[1:nsimu],s=30,c='red')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 180,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "nsimu = 21\n",
    "accuracy = [0]*nsimu\n",
    "ntree = [0]*nsimu\n",
    "for i in range(1,nsimu):\n",
    "    rf = RandomForestClassifier(n_estimators=i*5,min_samples_split=10,max_depth=10,criterion='entropy')\n",
    "    rf.fit(X_train,y_train)\n",
    "    rf_pred = rf.predict(X_test)\n",
    "    cm = confusion_matrix(y_test,rf_pred)\n",
    "    accuracy[i] = (cm[0,0]+cm[1,1])/cm.sum()\n",
    "    ntree[i]=i*5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 181,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA4AAAAGfCAYAAAAOK7wwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJzt3X+wnfV9J/a3VkIk3sTgKOuU+5FaSGA3xU5Miky8Tbezs6wpN42Rd0yMGDeGLVuYnZUdt0089u7QH56dKThpHXbNeIqBGNiscaKNG7mJyoxNdppkWhthHDfYyURmFUsfXBNfCGs7G4PS2z/OkTlcS9yDDnCFvq/XzJn7PN/zeb7388ycOdL7fp/znE2rq6sBAADg9PdXNroBAAAAXhoCIAAAwCAEQAAAgEEIgAAAAIMQAAEAAAYhAAIAAAxCAAQAABiEAAgAADAIARAAAGAQWza6gRfCa1/72tUdO3ZsdBsAAAAb4vOf//zXuvuvrVd3WgTAHTt2ZP/+/RvdBgAAwIaoqj+Zp84loAAAAIMQAAEAAAYhAAIAAAxCAAQAABiEAAgAADAIARAAAGAQAiAAAMAgBEAAAIBBCIAAAACDEAABAAAGsWWeoqq6PMktSTYnub27b1rz/JlJ7k5ycZKVJFd196GqemOSm5JsTfJUkp/v7vur6nuT/M7MFNuT/IvufldVXZvkF5L09LkPdvftJ3uCAAAATKwbAKtqc5Jbk7wxyZEkD1TVvu7+wkzZdUme6O7zq2p3kpuTXJXka0ne1N2PVtVrk9yXpLr760kumvkdDyb59Zn5PtbdexY8NwAAAGbMcwnoJUkOdvcj3f1UknuT7FpTsyvJXdPtvUkurapN3f1Qdz86HX84yXdNVwu/raouSPLqPHtFEAAAgBfYPAGwkhye2T8yHTtuTXcfTfJkkm1rat6S5KHu/taa8aszWfFbna2tqs9X1d6q2jFHjwAAAKxjngC46Thjq8+npqpek8lloTccp253ko/O7H8iybnd/aNJPplnVhafpaqur6oDVXVgZWXlOdoHAAAgmS8AHkkyuwq3PcmjJ6qpqi1Jzkry+HR/e5KPJ3l7d39p9qCqel2SLd394LGx7l6ZWSX8cCY3lvkO3X1bd+/s7p3btq1dbAQAAGCteQLgA0kuqKrzqmprJit2+9bU7EtyzXT7yiT3d/dqVZ2d5DeTvLe7f+84c1+dZ6/+parOmdm9IskX5+gRAACAdax7F9DuPlpVezK5g+fmJHd298NV9b4kB7p7X5I7ktxTVQczWfnbPT18T5Lzk9xYVTdOxy7r7sem229N8pNrfuU7q+qKJEenc1170mcHAADAt21aXV37cb6Xn+Xl5dX9+/dvdBsAAAAboqoe7O6d69XNcwkoAAAApwEBEAAAYBACIAAAwCAEQAAAgEEIgAAAAIMQAAEAAAYhAAIAAAxCAAQAABiEAAgAADAIARAAAGAQAiAAAMAgBEAAAIBBCIAAAACDEAABAAAGIQACAAAMQgAEAAAYhAAIAAAwCAEQAABgEAIgAADAIARAAACAQQiAAAAAgxAAAQAABiEAAgAADEIABAAAGIQACAAAMAgBEAAAYBACIAAAwCAEQAAAgEEIgAAAAIMQAAEAAAYhAAIAAAxCAAQAABiEAAgAADAIARAAAGAQAiAAAMAgBEAAAIBBbJmnqKouT3JLks1Jbu/um9Y8f2aSu5NcnGQlyVXdfaiq3pjkpiRbkzyV5Oe7+/7pMf86yTlJ/t10msu6+7ETzbXISQIAADDHCmBVbU5ya5LlJBcmubqqLlxTdl2SJ7r7/CQfSHLzdPxrSd7U3T+S5Jok96w57m3dfdH08dg6cwEAALCAeS4BvSTJwe5+pLufSnJvkl1ranYluWu6vTfJpVW1qbsf6u5Hp+MPJ/mu6QrfcznuXHP0CQAAwHOYJwBWksMz+0emY8et6e6jSZ5Msm1NzVuSPNTd35oZ++Wq+lxV3TgT8uaZK1V1fVUdqKoDKysrc5wGAADA2OYJgMdbfVt9PjVV9ZpMLuW8Yeb5t00vDf1b08fPPI/fl+6+rbt3dvfObdu+Ix8CAACwxjwB8EiSHTP725M8eqKaqtqS5Kwkj0/3tyf5eJK3d/eXjh3Q3T39+fUk/zKTS02fcy4AAABO3jwB8IEkF1TVeVW1NcnuJPvW1OzL5CYvSXJlkvu7e7Wqzk7ym0ne292/d6y4qrZU1fdPt89I8lNJ/uC55nr+pwYAAMCsdb8GoruPVtWeJPdl8jUQd3b3w1X1viQHuntfkjuS3FNVBzNZrds9PXxPkvOT3FhVN07HLkvyzST3TcPf5iSfTPLh6fMnmgsAAIAFbFpdffkvri0vL6/u379/o9sAAADYEFX1YHfvXK9unktAAQAAOA0IgAAAAIMQAAEAAAYhAAIAAAxCAAQAABiEAAgAADAIARAAAGAQAiAAAMAgBEAAAIBBCIAAAACDEAABAAAGIQACAAAMQgAEAAAYhAAIAAAwCAEQAABgEAIgAADAIARAAACAQQiAAAAAgxAAAQAABiEAAgAADEIABAAAGIQACAAAMAgBEAAAYBACIAAAwCAEQAAAgEEIgAAAAIMQAAEAAAYhAAIAAAxCAAQAABiEAAgAADAIARAAAGAQAiAAAMAgBEAAAIBBCIAAAACDEAABAAAGsWWeoqq6PMktSTYnub27b1rz/JlJ7k5ycZKVJFd196GqemOSm5JsTfJUkp/v7vur6hVJfi3JDyX5yySf6O73TOe6NskvJOnp9B/s7tsXOksAAADWXwGsqs1Jbk2ynOTCJFdX1YVryq5L8kR3n5/kA0luno5/LcmbuvtHklyT5J6ZY36xu384yY8l+YmqWp557mPdfdH0IfwBAAC8AOa5BPSSJAe7+5HufirJvUl2ranZleSu6fbeJJdW1abufqi7H52OP5zku6rqzO7+8+7+7SSZzvnZJNsXPRkAAABObJ4AWEkOz+wfmY4dt6a7jyZ5Msm2NTVvSfJQd3/rWQdWnZ3kTUk+NVtbVZ+vqr1VtWOOHgEAAFjHPAFw03HGVp9PTVW9JpPLQm+YLaiqLUk+muSfdfcj0+FPJDm3u380ySfzzMpi1hx7fVUdqKoDKysrc5wGAADA2OYJgEeSzK7CbU/y6IlqpqHurCSPT/e3J/l4krd395fWHHdbkj/u7l86NtDdKzOrhB/O5MYy36G7b+vund29c9u2tYuNAAAArDVPAHwgyQVVdV5VbU2yO8m+NTX7MrnJS5JcmeT+7l6dXt75m0ne292/N3tAVf3TTILiu9aMnzOze0WSL857MgAAAJzYul8D0d1Hq2pPkvsy+RqIO7v74ap6X5ID3b0vyR1J7qmqg5ms/O2eHr4nyflJbqyqG6djl2XytRD/JMkfJvlsVSXPfN3DO6vqiiRHp3Nd+4KcKQAAwOA2ra6u/Tjfy8/y8vLq/v37N7oNAACADVFVD3b3zvXq5rkEFAAAgNOAAAgAADAIARAAAGAQAiAAAMAgBEAAAIBBCIAAAACDEAABAAAGIQACAAAMQgAEAAAYhAAIAAAwCAEQAABgEAIgAADAIARAAACAQQiAAAAAgxAAAQAABiEAAgAADEIABAAAGIQACAAAMAgBEAAAYBACIAAAwCAEQAAAgEEIgAAAAIMQAAEAAAYhAAIAAAxCAAQAABiEAAgAADAIARAAAGAQAiAAAMAgBEAAAIBBCIAAAACDEAABAAAGIQACAAAMQgAEAAAYhAA4osOHk3e8I7nkksnPw4c3uqNn099i9LcY/S3uVO9Rf4vR32L0txj9LUZ/iznV+5vX6urqy/5x+eWXrzKnL395dfVVr1pdPeOM1dVk8vNVr5qMnwr0txj9LUZ/izvVe9TfYvS3GP0tRn+L0d9iTvX+VldXl5aWDqzOkZ2sAI7m/e9PvvGN5OmnJ/tPPz3Zf//7N7avY/S3GP0tRn+LO9V71N9i9LcY/S1Gf4vR32JO9f6ehy3zFFXV5UluSbI5ye3dfdOa589McneSi5OsJLmquw9V1RuT3JRka5Knkvx8d98/PebiJB9J8t1JfivJz3b3alV9X5KPJTk3yaEkb+3uJxY7Tb7t059+5oV7zNNPJ5/5zMb0s5b+FqO/xehvcad6j/pbjP4Wo7/F6G8x+lvMqd7f87DuCmBVbU5ya5LlJBcmubqqLlxTdl2SJ7r7/CQfSHLzdPxrSd7U3T+S5Jok98wc86Ek1ye5YPq4fDr+niSf6u4Lknxqus8L5cd/PDnjjGePnXHG5FrmU4H+FqO/xehvcad6j/pbjP4Wo7/F6G8x+lvMqd7f87HeNaJLS0t/c2lp6b6Z/fcuLS29d03NfUtLS39zur1laWnpa0tLS5vW1GxaWlpaWVpaOnNpaemcpaWlP5x57uqlpaX/dbr9R0tLS+dMt89ZWlr6o/V69BnA5+FUv35Zf4vR32L0t7hTvUf9LUZ/i9HfYvS3GP0t5lTvb/WF/QxgJZm9xc2R6dhxa7r7aJInk2xbU/OWJA9197em9UdOMOcPdPdXpnN9Jcmrj9tU1fVVdaCqDqysrMxxGiRJduxIfv/3kxtumPzF4oYbJvs7dmx0ZxP6W4z+FqO/xZ3qPepvMfpbjP4Wo7/F6G8xp3p/z8Om1dXV5yyoqp9O8p919z+Y7v9Mkku6+x0zNQ9Pa45M9780rVmZ7r8myb4kl3X3l6rq9Un+p+7+u9Pn/1aSd3f3m6rqz7r77Jm5n+juVz1Xj8vLy6v79+9/3icPAABwOqiqB7t753p186wAHkkyG223J3n0RDVVtSXJWUken+5vT/LxJG/v7i/N1G8/wZxfrapzpseek+SxOXoEAABgHfMEwAeSXFBV51XV1iS7M1nNm7Uvk5u8JMmVSe6f3tHz7CS/meS93f17x4qnl3Z+vareUFWbkrw9yW8cZ65rZsYBAABYwLoBcPqZvj1J7kvyxSS/2t0PV9X7quqKadkdSbZV1cEk/02euXPnniTnJ7mxqj43fRz7TN8/THJ7koNJvpTk2DWcNyV5Y1X9cZJjXyMBAADAgtb9DODLgc8AAgAAI3shPwMIAADAaUAABAAAGIQACAAAMAgBEAAAYBACIAAAwCAEQAAAgEEIgAAAAIMQAAEAAAYhAAIAAAxCAAQAABiEAAgAADAIARAAAGAQAiAAAMAgBEAAAIBBCIAAAACDEAABAAAGIQACAAAMQgAEAAAYhAAIAAAwCAEQAABgEAIgAADAIARAAACAQQiAAAAAgxAAAQAABiEAAgAADEIABAAAGIQACAAAMAgBEAAAYBACIAAAwCAEQAAAgEEIgAAAAIMQAAEAAAYhAAIAAAxCAAQAABiEAAgAADCILfMUVdXlSW5JsjnJ7d1905rnz0xyd5KLk6wkuaq7D1XVtiR7k7w+yUe6e8+0/nuT/M7MFNuT/IvufldVXZvkF5L09LkPdvftJ3l+AAAATK0bAKtqc5Jbk7wxyZEkD1TVvu7+wkzZdUme6O7zq2p3kpuTXJXkL5LcmOS100eSpLu/nuSimd/xYJJfn5nvY8fCIgAAAC+MeS4BvSTJwe5+pLufSnJvkl1ranYluWu6vTfJpVW1qbu/2d2/m0kQPK6quiDJq/PsFUEAAABeYPMEwEpyeGb/yHTsuDXdfTTJk0m2zdnD1Zms+K3OjL2lqj5fVXurasec8wAAAPAc5gmAm44ztnoSNSeyO8lHZ/Y/keTc7v7RJJ/MMyuLz1JV11fVgao6sLKyMuevAgAAGNc8AfBIktlVuO1JHj1RTVVtSXJWksfXm7iqXpdkS3c/eGysu1e6+1vT3Q9ncmOZ79Ddt3X3zu7euW3bvIuNAAAA45onAD6Q5IKqOq+qtmayYrdvTc2+JNdMt69Mcv+aSzpP5Oo8e/UvVXXOzO4VSb44xzwAAACsY927gHb30arak+S+TL4G4s7ufriq3pfkQHfvS3JHknuq6mAmK3+7jx1fVYeSvDLJ1qp6c5LLZu4g+tYkP7nmV76zqq5IcnQ617ULnB8AAABTm1ZX5/2o3qlreXl5df/+/RvdBgAAwIaoqge7e+d6dfNcAgoAAMBpQAAEAAAYhAAIAAAwCAEQAABgEAIgAADAIARAAACAQQiAAAAAgxAAAQAABiEAAgAADEIABAAAGIQACAAAMAgBEAAAYBACIAAAwCAEQAAAgEEIgAAAAIMQAAEAAAYhAAIAAAxCAAQAABiEAAgAADAIARAAAGAQAiAAAMAgBEAAAIBBCIAAAACDEAABAAAGIQACAAAMQgAEAAAYhAAIAAAwCAEQAABgEAIgAADAIARAAACAQQiAAAAAgxAAAQAABiEAAgAADEIAfDEcPpy84x3JJZdMfh4+vNEdAQAAZMtGN3DaOXw4ed3rkm98I3n66eRzn0t+5VeS3//9ZMeOje4OAAAYmBXAF9r73/9M+EsmP7/xjck4AADABpprBbCqLk9yS5LNSW7v7pvWPH9mkruTXJxkJclV3X2oqrYl2Zvk9Uk+0t17Zo7510nOSfLvpkOXdfdjJ5rrpM/wpfbpTz8T/o55+unkM5/ZmH4AAACm1l0BrKrNSW5NspzkwiRXV9WFa8quS/JEd5+f5ANJbp6O/0WSG5P83Ammf1t3XzR9PLbOXC8PP/7jyRlnPHvsjDMmnwcEAADYQPNcAnpJkoPd/Uh3P5Xk3iS71tTsSnLXdHtvkkuralN3f7O7fzeTIDiv4871PI7fWO9+d/I93/NMCDzjjMn+u9+9sX0BAADDmycAVpLZ21gemY4dt6a7jyZ5Msm2Oeb+5ar6XFXdOBPy5pqrqq6vqgNVdWBlZWWOX/US2bFjcsOXG26YrPrdcIMbwAAAAKeEeT4DeLzVt9WTqFnrbd3dVfW9Sf5Vkp/J5LN/c83V3bcluS1JlpeX1/tdL60dO5J//s83ugsAAIBnmWcF8EiS2eWr7UkePVFNVW1JclaSx59r0u7u6c+vJ/mXmVxqelJzAQAAsL55AuADSS6oqvOqamuS3Un2ranZl+Sa6faVSe7v7hOuylXVlqr6/un2GUl+KskfnMxcAAAAzGfdS0C7+2hV7UlyXyZfA3Fndz9cVe9LcqC79yW5I8k9VXUwk9W63ceOr6pDSV6ZZGtVvTnJZUn+JMl90/C3Ocknk3x4esgJ5wIAAODkbVpdffkvri0vL6/u379/o9sAAADYEFX1YHfvXK9unktAAQAAOA0IgAAAAIMQAAEAAAYhAAIAAAxCAAQAABiEAAgAADAIARAAAGAQAiAAAMAgBEAAAIBBCIAAAACDEAABAAAGIQACAAAMQgAEAAAYhAAIAAAwCAEQAABgEAIgAADAIARAAACAQQiAAAAAgxAAAQAABiEAAgAADEIABAAAGIQACAAAMAgBEAAAYBACIAAAwCAEQAAAgEEIgAAAAIMQAAEAAAYhAAIAAAxCAAQAABiEAAgAADAIARAAAGAQAiAAAMAgBEAAAIBBCIAAAACDEAABAAAGsWWeoqq6PMktSTYnub27b1rz/JlJ7k5ycZKVJFd196Gq2pZkb5LXJ/lId++Z1r8iya8l+aEkf5nkE939nulz1yb5hSQ9nf6D3X37IicJAADAHCuAVbU5ya1JlpNcmOTqqrpwTdl1SZ7o7vOTfCDJzdPxv0hyY5KfO87Uv9jdP5zkx5L8RFUtzzz3se6+aPoQ/gAAAF4A81wCekmSg939SHc/leTeJLvW1OxKctd0e2+SS6tqU3d/s7t/N5Mg+G3d/efd/dvT7aeSfDbJ9gXOAwAAgHXMEwAryeGZ/SPTsePWdPfRJE8m2TZPA1V1dpI3JfnUzPBbqurzVbW3qnbMMw8AAADPbZ4AuOk4Y6snUfMdqmpLko8m+Wfd/ch0+BNJzu3uH03yyTyzsrj22Our6kBVHVhZWVnvVwEAAAxvngB4JMnsKtz2JI+eqGYa6s5K8vgcc9+W5I+7+5eODXT3Snd/a7r74UxuLPMduvu27t7Z3Tu3bZtrsREAAGBo8wTAB5JcUFXnVdXWJLuT7FtTsy/JNdPtK5Pc393PuQJYVf80k6D4rjXj58zsXpHki3P0CAAAwDrW/RqI7j5aVXuS3JfJ10Dc2d0PV9X7khzo7n1J7khyT1UdzGTlb/ex46vqUJJXJtlaVW9OclmSf5vknyT5wySfrarkma97eGdVXZHk6HSua1+gcwUAABjaptXVdT+qd8pbXl5e3b9//0a3AQAAsCGq6sHu3rle3TyXgAIAAHAaEAABAAAGIQACAAAMQgAEAAAYhAAIAAAwCAEQAABgEAIgAADAIARAAACAQQiAAAAAgxAAAQAABiEAAgAADEIABAAAGIQACAAAMAgBEAAAYBACIAAAwCAEQAAAgEEIgAAAAIMQAAEAAAYhAAIAAAxCAAQAABiEAAgAADAIARAAAGAQAiAAAMAgBEAAAIBBCIAAAACDEAABAAAGIQACAAAMQgAEAAAYhAAIAAAwCAEQAABgEAIgAADAIARAAACAQQiAAAAAgxAAAQAABiEAAgAADEIABAAAGMSWeYqq6vIktyTZnOT27r5pzfNnJrk7ycVJVpJc1d2Hqmpbkr1JXp/kI929Z+aYi5N8JMl3J/mtJD/b3atV9X1JPpbk3CSHkry1u59Y4BwBAADIHCuAVbU5ya1JlpNcmOTqqrpwTdl1SZ7o7vOTfCDJzdPxv0hyY5KfO87UH0pyfZILpo/Lp+PvSfKp7r4gyaem+wAAACxonktAL0lysLsf6e6nktybZNeaml1J7ppu701yaVVt6u5vdvfvZhIEv62qzknyyu7+v7p7NZPVwzcfZ667ZsYBAABYwDwBsJIcntk/Mh07bk13H03yZJJt68x55ARz/kB3f2U611eSvPq4E1RdX1UHqurAysrKHKcBAAAwtnk+A7jpOGOrJ1GzSP136O7bktyWJMvLy8/rWAAAgBHNswJ4JMmOmf3tSR49UU1VbUlyVpLH15lz+wnm/Or0EtFjl4o+NkePAAAArGOeAPhAkguq6ryq2ppkd5J9a2r2Jblmun1lkvunn+07rumlnV+vqjdU1aYkb0/yG8eZ65qZcQAAABaw7iWg3X20qvYkuS+Tr4G4s7sfrqr3JTnQ3fuS3JHknqo6mMnK3+5jx1fVoSSvTLK1qt6c5LLu/kKSf5hnvgZi//SRJDcl+dWqui7Jl5P89AtxogAAAKPbtLr68v/43PLy8ur+/fvXLwQAADgNVdWD3b1zvbp5LgEFAADgNCAAAgAADEIABAAAGIQACAAAMAgBEAAAYBACIAAAwCAEQAAAgEEIgAAAAIMQAAEAAAYhAAIAAAxCAAQAABiEAAgAADAIARAAAGAQAiAAAMAgBEAAAIBBCIAAAACDEAABAAAGIQACAAAMQgAEAAAYhAAIAAAwCAEQAABgEAIgAADAIARAAACAQQiAAAAAgxAAAQAABiEAAgAADEIABAAAGIQACAAAMAgBEAAAYBACIAAAwCAEQAAAgEEIgAAAAIMQAAEAAAYhAAIAAAxCAAQAABjElnmKquryJLck2Zzk9u6+ac3zZya5O8nFSVaSXNXdh6bPvTfJdUn+Msk7u/u+qvobST42M8UPJvnvuvuXqup/SPJfJfnT6XP/uLt/6+RODwAAgGPWDYBVtTnJrUnemORIkgeqal93f2Gm7LokT3T3+VW1O8nNSa6qqguT7E7ymiRLST5ZVX+9u/8oyUUz83eSj8/M94Hu/sXFTw8AAIBj5rkE9JIkB7v7ke5+Ksm9SXatqdmV5K7p9t4kl1bVpun4vd39re7+N0kOTuebdWmSL3X3n5zsSQAAALC+eQJgJTk8s39kOnbcmu4+muTJJNvmPHZ3ko+uGdtTVZ+vqjur6lVz9AgAAMA65gmAm44ztjpnzXMeW1Vbk1yR5Ndmnv9Qkh/K5BLRryT5n4/XVFVdX1UHqurAysrKibsHAAAgyXw3gTmSZMfM/vYkj56g5khVbUlyVpLH5zh2Oclnu/urxwZmt6vqw0n+9+M11d23JbktSZaXl9cGUgAAANaYZwXwgSQXVNV50xW73Un2ranZl+Sa6faVSe7v7tXp+O6qOrOqzktyQZLPzBx3ddZc/llV58zs/r0kfzDvyQAAAHBi664AdvfRqtqT5L5Mvgbizu5+uKrel+RAd+9LckeSe6rqYCYrf7unxz5cVb+a5AtJjib5R939l0lSVa/I5M6iN6z5le+vqosyuVT00HGeBwAA4CRsWl19+V89uby8vLp///6NbgMAAGBDVNWD3b1zvbp5LgEFAADgNCAAAgAADEIABAAAGIQACAAAMIjT4iYwVfWnSf5ko/tgON+f5Gsb3QTD8vpjo3kNspG8/thIp+rr7z/o7r+2XtFpEQBhI1TVgXnutAQvBq8/NprXIBvJ64+N9HJ//bkEFAAAYBACIAAAwCAEQDh5t210AwzN64+N5jXIRvL6YyO9rF9/PgMIAAAwCCuAAAAAg9iy0Q3Ay0FV7Uhyd5J/L8n/l+S27r6lqr4vyceSnJvkUJK3dvcTG9Unp7eq2pzkQJLu7p+qqvOS3Jvk+5J8NsnPdPdTG9kjp6eqOjvJ7Ulem2Q1yX+Z5I/i/Y+XQFX910n+QSavvf8nyd9Pck68//Eiqao7k/xUkse6+7XTseP+n6+qNiW5JclPJvnzJNd292c3ou95WQGE+RxN8t9293+Y5A1J/lFVXZjkPUk+1d0XJPnUdB9eLD+b5Isz+zcn+cD09fdEkus2pCtGcEuS/6O7fzjJ6zJ5HXr/40VXVZXknUl2Tv8jvjnJ7nj/48X1kSSXrxk70XvecpILpo/rk3zoJerxpAmAMIfu/sqxv+Z099cz+c9PJdmV5K5p2V1J3rwxHXK6q6rtSf7zTFZhMv2L499Jsnda4vXHi6KqXpnkP01yR5J091Pd/Wfx/sdLZ0uS766qLUlekeQr8f7Hi6i7/88kj68ZPtF73q4kd3f3anf/30nOrqpzXppOT44ACM9TVZ2b5MeSfDrJD3T3V5JJSEzy6g1sjdPbLyV5dyaXICfJtiR/1t1Hp/tHMvmjBLzQfjDJnyb55ap6qKpur6q/Gu9/vAS6u5P8YpIvZxL8nkzyYLz/8dI70XvM7J1uAAACAklEQVReJTk8U3fKvx4FQHgequp7kvyrJO/q7n+70f0whqo69jmEB2eGNx2n1G2deTFsSfIfJflQd/9Ykm/G5Z68RKrqVZmssJyXZCnJX83kkru1vP+xUV52/x4LgDCnqjojk/D3K93969Phrx5b5p/+fGyj+uO09hNJrqiqQ5nc9ODvZLIiePb0kqgk2Z7k0Y1pj9PckSRHuvvT0/29mQRC73+8FP5ukn/T3X/a3U8n+fUk/3G8//HSO9F73pEkO2bqTvnXowAIc5h+3uqOJF/s7v9l5ql9Sa6Zbl+T5Dde6t44/XX3e7t7e3efm8nND+7v7rcl+e0kV07LvP54UXT3/5vkcFX9jenQpUm+EO9/vDS+nOQNVfWK6b/Fx15/3v94qZ3oPW9fkrdX1aaqekOSJ49dKnqq8kXwMIeq+k+S/E4mt58+9hmsf5zJ5wB/Ncm/n8k/Uj/d3Ws/NAwvmKr620l+bvo1ED+YZ26D/lCS/6K7v7WR/XF6qqqLMrkB0dYkj2RyG/6/Eu9/vASq6n9MclUmd+R+KJOvhKh4/+NFUlUfTfK3k3x/kq8m+e+T/G85znve9A8TH8zkrqF/nuTvd/eBjeh7XgIgAADAIFwCCgAAMAgBEAAAYBACIAAAwCAEQAAAgEEIgAAAAIMQAAEAAAYhAAIAAAxCAAQAABjE/w9lJmgVI+LMwQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1124f9a58>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(15,7))\n",
    "plt.scatter(x=ntree[1:nsimu],y=accuracy[1:nsimu],s=30,c='red')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 182,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.cross_validation import ShuffleSplit\n",
    "from sklearn.grid_search import GridSearchCV\n",
    "def fit_model(X,y):\n",
    "    cv_sets = ShuffleSplit(X_train.shape[0],n_iter=5,test_size=0.20,random_state=1234)\n",
    "    nnt = MLPClassifier()\n",
    "    params = {'activation':('relu','logistic'),\n",
    "              'hidden_layer_sizes':[2,3],\n",
    "              'solver':('sgd','adam','lbfgs'),\n",
    "             'max_iter' : [200,500],\n",
    "             'alpha' : [0.0001,0.001,0.01]}\n",
    "    grid = GridSearchCV(nnt,params,cv=cv_sets)\n",
    "    grid = grid.fit(X,y)\n",
    "    return grid.best_estimator_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 172,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#Next Step:\n",
    "# Grid search for hyper paramter tuning\n",
    "# TF - models\n",
    "# CNN- for image classification"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 350,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Populating the interactive namespace from numpy and matplotlib\n"
     ]
    }
   ],
   "source": [
    "## Importing standard libraries\n",
    "%pylab inline\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#https://s3.amazonaws.com/hackerday.datascience/118/train.csv\n",
    "#https://s3.amazonaws.com/hackerday.datascience/118/test.csv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 352,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "## Measure execution time, becaus Kaggle cloud fluctuates  \n",
    "import time\n",
    "start = time.time()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 353,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "## Importing sklearn libraries\n",
    "\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.cross_validation import train_test_split\n",
    "from sklearn.preprocessing import LabelEncoder\n",
    "from sklearn.model_selection import StratifiedShuffleSplit\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 354,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "## Keras Libraries for Neural Networks\n",
    "from keras.models import Sequential\n",
    "from keras.layers import Merge\n",
    "from keras.layers import Dense, Dropout, Activation, Flatten\n",
    "from keras.layers.advanced_activations import PReLU\n",
    "from keras.layers import Convolution2D, MaxPooling2D\n",
    "from keras.utils.np_utils import to_categorical\n",
    "from keras.callbacks import EarlyStopping"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 355,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "## Read data from the CSV file\n",
    "data = pd.read_csv('https://s3.amazonaws.com/hackerday.datascience/118/train.csv')\n",
    "parent_data = data.copy()    ## Always a good idea to keep a copy of original data\n",
    "ID = data.pop('id')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 356,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>margin1</th>\n",
       "      <th>margin2</th>\n",
       "      <th>margin3</th>\n",
       "      <th>margin4</th>\n",
       "      <th>margin5</th>\n",
       "      <th>margin6</th>\n",
       "      <th>margin7</th>\n",
       "      <th>margin8</th>\n",
       "      <th>margin9</th>\n",
       "      <th>margin10</th>\n",
       "      <th>...</th>\n",
       "      <th>texture55</th>\n",
       "      <th>texture56</th>\n",
       "      <th>texture57</th>\n",
       "      <th>texture58</th>\n",
       "      <th>texture59</th>\n",
       "      <th>texture60</th>\n",
       "      <th>texture61</th>\n",
       "      <th>texture62</th>\n",
       "      <th>texture63</th>\n",
       "      <th>texture64</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>990.000000</td>\n",
       "      <td>990.000000</td>\n",
       "      <td>990.000000</td>\n",
       "      <td>990.000000</td>\n",
       "      <td>990.000000</td>\n",
       "      <td>990.000000</td>\n",
       "      <td>990.000000</td>\n",
       "      <td>990.000000</td>\n",
       "      <td>990.000000</td>\n",
       "      <td>990.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>990.000000</td>\n",
       "      <td>990.000000</td>\n",
       "      <td>990.000000</td>\n",
       "      <td>990.000000</td>\n",
       "      <td>990.000000</td>\n",
       "      <td>990.000000</td>\n",
       "      <td>990.000000</td>\n",
       "      <td>990.000000</td>\n",
       "      <td>990.000000</td>\n",
       "      <td>990.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.017412</td>\n",
       "      <td>0.028539</td>\n",
       "      <td>0.031988</td>\n",
       "      <td>0.023280</td>\n",
       "      <td>0.014264</td>\n",
       "      <td>0.038579</td>\n",
       "      <td>0.019202</td>\n",
       "      <td>0.001083</td>\n",
       "      <td>0.007167</td>\n",
       "      <td>0.018639</td>\n",
       "      <td>...</td>\n",
       "      <td>0.036501</td>\n",
       "      <td>0.005024</td>\n",
       "      <td>0.015944</td>\n",
       "      <td>0.011586</td>\n",
       "      <td>0.016108</td>\n",
       "      <td>0.014017</td>\n",
       "      <td>0.002688</td>\n",
       "      <td>0.020291</td>\n",
       "      <td>0.008989</td>\n",
       "      <td>0.019420</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.019739</td>\n",
       "      <td>0.038855</td>\n",
       "      <td>0.025847</td>\n",
       "      <td>0.028411</td>\n",
       "      <td>0.018390</td>\n",
       "      <td>0.052030</td>\n",
       "      <td>0.017511</td>\n",
       "      <td>0.002743</td>\n",
       "      <td>0.008933</td>\n",
       "      <td>0.016071</td>\n",
       "      <td>...</td>\n",
       "      <td>0.063403</td>\n",
       "      <td>0.019321</td>\n",
       "      <td>0.023214</td>\n",
       "      <td>0.025040</td>\n",
       "      <td>0.015335</td>\n",
       "      <td>0.060151</td>\n",
       "      <td>0.011415</td>\n",
       "      <td>0.039040</td>\n",
       "      <td>0.013791</td>\n",
       "      <td>0.022768</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.001953</td>\n",
       "      <td>0.001953</td>\n",
       "      <td>0.013672</td>\n",
       "      <td>0.005859</td>\n",
       "      <td>0.001953</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.005859</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.001953</td>\n",
       "      <td>0.005859</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000977</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.004883</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000977</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.009766</td>\n",
       "      <td>0.011719</td>\n",
       "      <td>0.025391</td>\n",
       "      <td>0.013672</td>\n",
       "      <td>0.007812</td>\n",
       "      <td>0.015625</td>\n",
       "      <td>0.015625</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.005859</td>\n",
       "      <td>0.015625</td>\n",
       "      <td>...</td>\n",
       "      <td>0.004883</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.005859</td>\n",
       "      <td>0.000977</td>\n",
       "      <td>0.012695</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.003906</td>\n",
       "      <td>0.002930</td>\n",
       "      <td>0.011719</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.025391</td>\n",
       "      <td>0.041016</td>\n",
       "      <td>0.044922</td>\n",
       "      <td>0.029297</td>\n",
       "      <td>0.017578</td>\n",
       "      <td>0.056153</td>\n",
       "      <td>0.029297</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.007812</td>\n",
       "      <td>0.027344</td>\n",
       "      <td>...</td>\n",
       "      <td>0.043701</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.022217</td>\n",
       "      <td>0.009766</td>\n",
       "      <td>0.021484</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.023438</td>\n",
       "      <td>0.012695</td>\n",
       "      <td>0.029297</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>0.087891</td>\n",
       "      <td>0.205080</td>\n",
       "      <td>0.156250</td>\n",
       "      <td>0.169920</td>\n",
       "      <td>0.111330</td>\n",
       "      <td>0.310550</td>\n",
       "      <td>0.091797</td>\n",
       "      <td>0.031250</td>\n",
       "      <td>0.076172</td>\n",
       "      <td>0.097656</td>\n",
       "      <td>...</td>\n",
       "      <td>0.429690</td>\n",
       "      <td>0.202150</td>\n",
       "      <td>0.172850</td>\n",
       "      <td>0.200200</td>\n",
       "      <td>0.106450</td>\n",
       "      <td>0.578130</td>\n",
       "      <td>0.151370</td>\n",
       "      <td>0.375980</td>\n",
       "      <td>0.086914</td>\n",
       "      <td>0.141600</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>8 rows × 192 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "          margin1     margin2     margin3     margin4     margin5     margin6  \\\n",
       "count  990.000000  990.000000  990.000000  990.000000  990.000000  990.000000   \n",
       "mean     0.017412    0.028539    0.031988    0.023280    0.014264    0.038579   \n",
       "std      0.019739    0.038855    0.025847    0.028411    0.018390    0.052030   \n",
       "min      0.000000    0.000000    0.000000    0.000000    0.000000    0.000000   \n",
       "25%      0.001953    0.001953    0.013672    0.005859    0.001953    0.000000   \n",
       "50%      0.009766    0.011719    0.025391    0.013672    0.007812    0.015625   \n",
       "75%      0.025391    0.041016    0.044922    0.029297    0.017578    0.056153   \n",
       "max      0.087891    0.205080    0.156250    0.169920    0.111330    0.310550   \n",
       "\n",
       "          margin7     margin8     margin9    margin10     ...       texture55  \\\n",
       "count  990.000000  990.000000  990.000000  990.000000     ...      990.000000   \n",
       "mean     0.019202    0.001083    0.007167    0.018639     ...        0.036501   \n",
       "std      0.017511    0.002743    0.008933    0.016071     ...        0.063403   \n",
       "min      0.000000    0.000000    0.000000    0.000000     ...        0.000000   \n",
       "25%      0.005859    0.000000    0.001953    0.005859     ...        0.000000   \n",
       "50%      0.015625    0.000000    0.005859    0.015625     ...        0.004883   \n",
       "75%      0.029297    0.000000    0.007812    0.027344     ...        0.043701   \n",
       "max      0.091797    0.031250    0.076172    0.097656     ...        0.429690   \n",
       "\n",
       "        texture56   texture57   texture58   texture59   texture60   texture61  \\\n",
       "count  990.000000  990.000000  990.000000  990.000000  990.000000  990.000000   \n",
       "mean     0.005024    0.015944    0.011586    0.016108    0.014017    0.002688   \n",
       "std      0.019321    0.023214    0.025040    0.015335    0.060151    0.011415   \n",
       "min      0.000000    0.000000    0.000000    0.000000    0.000000    0.000000   \n",
       "25%      0.000000    0.000977    0.000000    0.004883    0.000000    0.000000   \n",
       "50%      0.000000    0.005859    0.000977    0.012695    0.000000    0.000000   \n",
       "75%      0.000000    0.022217    0.009766    0.021484    0.000000    0.000000   \n",
       "max      0.202150    0.172850    0.200200    0.106450    0.578130    0.151370   \n",
       "\n",
       "        texture62   texture63   texture64  \n",
       "count  990.000000  990.000000  990.000000  \n",
       "mean     0.020291    0.008989    0.019420  \n",
       "std      0.039040    0.013791    0.022768  \n",
       "min      0.000000    0.000000    0.000000  \n",
       "25%      0.000000    0.000000    0.000977  \n",
       "50%      0.003906    0.002930    0.011719  \n",
       "75%      0.023438    0.012695    0.029297  \n",
       "max      0.375980    0.086914    0.141600  \n",
       "\n",
       "[8 rows x 192 columns]"
      ]
     },
     "execution_count": 356,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.shape\n",
    "data.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 357,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(990,)\n"
     ]
    }
   ],
   "source": [
    "## Since the labels are textual, so we encode them categorically\n",
    "\n",
    "y = data.pop('species')\n",
    "y = LabelEncoder().fit(y).transform(y)\n",
    "print(y.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 358,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(990, 192)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([[-0.48661074, -0.13135701, -0.33095592, ..., -0.39487147,\n",
       "        -0.65214311,  0.26239707],\n",
       "       [-0.58560191, -0.73488047, -0.02856115, ..., -0.49497401,\n",
       "         2.18166976,  0.13364087],\n",
       "       [-0.58560191, -0.48340807, -0.48219201, ..., -0.52001246,\n",
       "         0.83564129, -0.72463124],\n",
       "       ..., \n",
       "       [-0.78358426, -0.63430181, -1.23821765, ...,  0.1807566 ,\n",
       "        -0.65214311, -0.76756462],\n",
       "       [-0.88257543, -0.73488047,  0.5762671 , ..., -0.52001246,\n",
       "        -0.51045972, -0.72463124],\n",
       "       [ 0.30542003, -0.23196142, -0.02856115, ...,  0.08065405,\n",
       "         1.18988604,  0.13364087]])"
      ]
     },
     "execution_count": 358,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "## Most of the learning algorithms are prone to feature scaling\n",
    "## Standardising the data to give zero mean =)\n",
    "from sklearn import preprocessing\n",
    "X = preprocessing.MinMaxScaler().fit(data).transform(data)\n",
    "X = StandardScaler().fit(data).transform(data)\n",
    "## normalizing does not help here; l1 and l2 allowed\n",
    "## X = preprocessing.normalize(data, norm='l1')\n",
    "print(X.shape)\n",
    "X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 359,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(990, 99)\n"
     ]
    }
   ],
   "source": [
    "## We will be working with categorical crossentropy function\n",
    "## It is required to further convert the labels into \"one-hot\" representation\n",
    "\n",
    "y_cat = to_categorical(y)\n",
    "print(y_cat.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 360,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x_train dim:  (792, 192)\n",
      "x_val dim:    (198, 192)\n"
     ]
    }
   ],
   "source": [
    "## retain class balances\n",
    "sss = StratifiedShuffleSplit(n_splits=10, test_size=0.2,random_state=12345)\n",
    "train_index, val_index = next(iter(sss.split(X, y)))\n",
    "x_train, x_val = X[train_index], X[val_index]\n",
    "y_train, y_val = y_cat[train_index], y_cat[val_index]\n",
    "print(\"x_train dim: \",x_train.shape)\n",
    "print(\"x_val dim:   \",x_val.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 361,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "## Developing a layered model for Neural Networks\n",
    "## Input dimensions should be equal to the number of features\n",
    "## We used softmax layer to predict a uniform probabilistic distribution of outcomes\n",
    "## https://keras.io/initializations/ ;glorot_uniform, glorot_normal, lecun_uniform, orthogonal,he_normal\n",
    "# DNN- Deep Neural Network\n",
    "model = Sequential()\n",
    "model.add(Dense(768,input_dim=192,  init='glorot_normal', activation='tanh'))\n",
    "model.add(Dropout(0.4))\n",
    "\n",
    "model.add(Dense(768, activation='tanh'))\n",
    "model.add(Dropout(0.4))\n",
    "\n",
    "model.add(Dense(99, activation='softmax'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 371,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "## Error is measured as categorical crossentropy or multiclass logloss\n",
    "## Adagrad, rmsprop, SGD, Adadelta, Adam, Adamax, Nadam\n",
    "\n",
    "#model.compile(loss='categorical_crossentropy',optimizer='rmsprop', metrics = [\"accuracy\"])\n",
    "model.compile(loss='categorical_crossentropy',optimizer='adagrad', metrics = [\"accuracy\"])\n",
    "#model.compile(loss='categorical_crossentropy',optimizer='sgd', metrics = [\"accuracy\"])\n",
    "#model.compile(loss='categorical_crossentropy',optimizer='adadelta', metrics = [\"accuracy\"])\n",
    "#model.compile(loss='categorical_crossentropy',optimizer='adam', metrics = [\"accuracy\"])\n",
    "#model.compile(loss='categorical_crossentropy',optimizer='adamax', metrics = [\"accuracy\"])\n",
    "#model.compile(loss='categorical_crossentropy',optimizer='Nadam', metrics = [\"accuracy\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 372,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "## Fitting the model on the whole training data with early stopping\n",
    "early_stopping = EarlyStopping(monitor='val_loss', patience=100)\n",
    "\n",
    "history = model.fit(x_train, y_train,batch_size=192,nb_epoch=250 ,verbose=0,\n",
    "                    validation_data=(x_val, y_val),callbacks=[early_stopping])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 373,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "val_acc:  0.994949514216\n",
      "val_loss:  0.0375406473706\n",
      "train_acc:  1.0\n",
      "train_loss:  7.1233824642e-06\n",
      "\n",
      "train/val loss ratio:  0.000189751188728\n"
     ]
    }
   ],
   "source": [
    "## we need to consider the loss for final submission to leaderboard\n",
    "## print(history.history.keys())\n",
    "print('val_acc: ',max(history.history['val_acc']))\n",
    "print('val_loss: ',min(history.history['val_loss']))\n",
    "print('train_acc: ',max(history.history['acc']))\n",
    "print('train_loss: ',min(history.history['loss']))\n",
    "\n",
    "print()\n",
    "print(\"train/val loss ratio: \", min(history.history['loss'])/min(history.history['val_loss']))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 374,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEWCAYAAABxMXBSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJzsnXmcXGWV97+31u7qfU/ydGcjgSSEfQdFQEAiIIy4KyPqCLwjM868o+M64yw6+I7jzKgwKjqIMgoijkgExAUIW4CEJSELCdk63TdLd1fvtS/3/eMuVdVd3V1dXZ1OKuf7+QSqa7n3qe6q53d/55znPJphGAiCIAhCobjmegCCIAjCsYUIhyAIgjAtRDgEQRCEaSHCIQiCIEwLEQ5BEARhWohwCIIgCNNChEMQSohS6h6l1FcLfO4+pdTlMz2OIBxpRDgEQRCEaSHCIQiCIEwLz1wPQBCONEqpfcCdwI3ACcD9wBeBe4C3AC8C79V1fcB6/ruA2wEFvAb8H13Xt1uPnQH8N7AceBTIacWglLoG+CqwGNgG3Krr+uYixvxJ4HNAI/CsdZwDSikN+Hfgw4Af6AQ+pOv6FqXUO4F/AzqAYeA/dF3/t+meWxDGIo5DOF65AbgCOBG4FngMUzyaMb8XfwmglDoRuA/4K6AFUxzWKqV8Sikf8BBwL+aE/gvruFivPRO4G7gFaAK+DzyslPJPZ6BKqcswhet9wHxMcbjfevhK4GLrfdQD7weC1mP/Ddyi63oNsBp4YjrnFYSJEMchHK98R9f1wwBKqWeAHl3XX7V+/hXwdut57wce0XX999Zj/wZ8GrgQSANe4D91XTeAB5VS/zfrHJ8Evq/r+ovWzz9WSn0ROB9YN42xfhi4W9f1V6wxfAEYUEotBhJADbACeMl2QhYJYJVSapPlngamcU5BmBBxHMLxyuGs25E8P1dbtxdgXuEDoOt6GujCDFstMO/Ss8NTnVm3FwF/o5QatP9hho0WTHOsY8cwiukqlK7rTwB3YIbeDiul7lJK1VpPvQF4J9CplFqnlLpgmucVhLyIcAjC5BzAFAAArJxCB6ADB827lJb1/IVZt7uAr+m6Xp/1L6Dr+n0zHEMVZuhLB9B1/du6rp8FnIwZsvqsdf8GXdevA1oxQ2oPTPO8gpAXCVUJwuQ8AHxeKfV24GnMMFUMeN56PAn8pVLqTuBdwLnAk9ZjPwB+pZT6A/ASEAAuAZ7WdX1kGmP4GXC/UupnwHbgX4AXdV3fp5Q6B/MC8BUgBESBlJV/eS/wG13Xh5RSw0CqmF+AIIxFHIcgTIKu6zuAjwDfAfowE+nX6roe13U9DrwbuAkzf/B+4H+zXrsRM89xh/X4Luu50x3DH4G/A36J6XJOAD5gPVyLKVADmOGsIGYlFZhVY/ss0bjVeh+CMGM02chJEARBmA7iOARBEIRpIcIhCIIgTAsRDkEQBGFaiHAIgiAI06Isy3FXr15tdHR0zPUwBEEQjik2b97cp+t6y1TPK0vh6Ojo4LHHHpvrYQiCIBxTKKU6p37WMSAc1irZ/wLiwFO6rv90jockCIJwXDMnOQ6l1N1KqR6l1JYx91+llNqhlNqllPq8dfe7gQd1Xf8k5spcQRAEYQ6Zq+T4PcBV2XcopdyYjdrWAKuADyqlVgHtmD1/QFomCIIgzDlzEqrSdf1pqyV0NucCu3Rd3wOglLofuA7oxhSP15hE6JRSNwM3W7fHPZ5IJOju7iYajZbiLRy1VFRU0N7ejtfrneuhCIJQphxNOQ5FxlmAKRjnAd8G7lBKXQ2snejFuq7fBdwFsGbNmnF9VLq7u6mpqWHx4sVomjbu9eWAYRgEg0G6u7tZsmTJXA9HEIQy5WgSjnyzuaHregj4WCEHUEpdC1y7ePHicY9Fo9GyFg0ATdNoamqit7d3rociCEIZczQtAOzG3OfAph1zH4KC0XV9ra7rN9fW1uZ9vJxFw+Z4eI+CIMwtR5Pj2AAsV0otwdyg5gPAh6ZzgMkcR0FEBiEVg+q24l4vCIJwHDBX5bj3AeuBk5RS3UqpT+i6ngRuAx7H3KzmAV3Xt07nuFM5jimJDcNoT3GvnYKhoSHuueeeab/uxhtvZGhoqPQDEgRBKJK5qqr64AT3Pwo8WuxxZ+w4XF5IJ8FIg1ZaTR0eHuYnP/kJN910U879qVQKt9s94evuvffeko5DEARhphxNoaoZo+v6WmDtmjVrPlnUAdxWCWsqCR5f6QYG/Mu//AudnZ1cccUVeL1eAoEAbW1tbN26laeeeoqPf/zjHDhwgFgsxic+8Qk+8hFzs7bzzjuPxx57jFAoxEc+8hHOPfdcNm7cyLx587j77ruprKws6TgFQRCmoqyEo1D+ce1Wth0YHnd/KpnAnY6B9yXQJnYB+Vi1oJavXHvyhI9/8YtfZMeOHfz+97/n+eef50//9E954oknWLhwIQDf/OY3aWhoIBKJcPXVV/POd76TxsbGnGPs3buXO++8k2984xvccsstPProo9xwww3TGqcgCMJMKSvhmGmoKo2GG8Aw8hcHl5DTTz/dEQ2Au+++22nMeODAAfbu3TtOODo6Oli9ejUAp556Kl1dXQiCIBxpyko4Cg1VTeQM+oZGaQ69Sbq2HVf1lJ2FZ0QgEHBuP//88zzzzDOsXbuWyspK3vOe9xCLxca9xu/3O7fdbnfZr4IXBOHo5GhaxzHnuD1eDAPSyXjJj11VVcXo6Gjex0ZGRqirq6OyspJdu3bxyiuvlPz8giAIpaKsHMdMQ1Vej5sEbrRkoqTjAmhsbOScc87hsssuo6KigubmZuexSy65hHvvvZfLL7+cpUuXcuaZZ5b8/IIgCKVCM4xxbZ2OedasWWOM3chp+/btrFy5ctLXxZIpUod34PV68LaeOJtDnFUKea+CIAhjUUq9rOv62VM9T0JVWXhdLhK4zbUcgiAIQl5EOLJwuTRSmgeXCIcgCMKESI5jDGnNi9tIQToNLtFVQRCEsZTVzDjjXlUAbktL06VPkAuCIJQDZSUcJcFltx0R4RAEQciHCMcYXFaPqnSq9Gs5BEEQygERjjHYwpEq8VqOYtuqA/zgBz8gEomUdDyCIAjFUlbCoZS6Vil11/Dw+AaGheLxmDmOdKq0lVV2W/Vi+OEPfyjCIQjCUUNZVVXNuK064PW4SBsaRjpVwpHltlW/+OKLaW5uZu3atcTjca666io+85nPEA6HueWWWzh48CDpdJpPf/rT9PX1cfjwYd773vfS0NDAgw8+WNJxCYIgTJeyEo6CeezzcOj1vA/5MCAewqt5wFtR+DHnnQJrvj7hw9lt1detW8cjjzzCI488gmEY3HTTTbzwwgsEg0HmzZvnbN40PDxMbW0td911F7/4xS/GdcsVBEGYC8oqVFUqjKz/zgbr1q1j3bp1XHnllbzjHe9g9+7d7N27lxUrVvDMM8/wta99jRdffJEZlRULgiDMEsen45jEGWhA7MBWDLePyrbls3J6wzC47bbbuPHGG8c99thjj/HEE09w++2387a3vY2//uu/npUxCIIgFIs4jjykcaEZ6ZIeM7ut+iWXXMLPf/5zQqEQAAcPHqSvr49Dhw5RWVnJDTfcwK233srrr5vhtOrq6glbsguCIBxpyspxlKLlCIAxC8KR3Vb90ksv5frrr+dd73oXYG7q9J3vfId9+/bx1a9+FU3T8Hq93H777QB8+MMf5iMf+Qitra2SHBcEYc6Rtup5GD34Jj4jjm/BxHuIH81IW3VBEIpB2qrPAENzoVFaxyEIglAuiHDkQ3PhEuEQBEHIy3ElHAWH5TS3KRzHYBivHEOPgiAcXRw3wlFRUUEwGCxsYtVcaIBR4gT5bGMYBsFgkIqKaSxcFARBmCZlVVU1Ge3t7XR3d9Pb2zvlc6OjQ1QkhzAGtqO53EdgdKWjoqKC9vb2uR6GIAhlzHEjHF6vlyVLlhT03Oce/DZnbPk7gp94kaaOFbM8MkEQhGOL4yZUNR08lWarj8jI4ByPRBAE4ejjqHccSqmlwJeAOl3X33Mkzum1hCMaGjkSpxMEQTimmFXHoZS6WynVo5TaMub+q5RSO5RSu5RSn5/sGLqu79F1/ROzOc6x+KtM4YiFh47kaQVBEI4JZttx3APcATg7GCml3MCdwBVAN7BBKfUw4AZuH/P6j+u63jPLYxxHhSUciXDxG0IJgiCUK7MqHLquP62UWjzm7nOBXbqu7wFQSt0PXKfr+u3ANcWeSyl1M3CzdbvYwwBQWV0HQCIiwiEIgjCWuUiOK6Ar6+du6778T1aqSSn1PeAMpdQXJnqerut36bp+tq7rZzc1Nc1ogFU19QCkotKRVhAEYSxzkRzX8tw34ao8XdeDwK2FHLhU3XGrakzHkY5KclwQBGEsc+E4uoGOrJ/bgQOlOLCu62t1Xb95pjvneXwVxA0PxMVxCIIgjGUuHMcGYLlSagmgAx8APlSKA5fKcQCEtUo0EQ5BKD/stkNavuCHUAizXY57H7AeOEkp1a2U+oSu60ngNuBxYDvwgK7rW0txvlI5DoCIVokrESrBqARBOKr44eXw5NfmehTHNLNdVfXBCe5/FHi01OcrpeOIuQK4kyIcglBWJKKgvwyaCy778lyP5pilrFqOlNJxJNwBvMlwCUYlCMJRw8BewIDDWyF9bHW/PpooK+EoJQlPAF9KhEMQyorgbvP/iRAM7pvToRzLlJVwKKWuVUrdNTw884V7SU8VfkOEQxDKiuCuzO1DWyZ+njApZSUcpQxVpb3VVKYjJRiVIAhHDcFdUFFn5jgOl6QmZ+Z0vQSvPwhvPAq7n4Sh7rke0ZQc9d1x5wrDV0UVYYy9z6C1nQyBxrkekiAIM6V/D7SuglAfHD4KHMfWh+AXHx1/f/1CqFXmv+bl4KkwxU7TwOWx/rmt/3tzf150EVTNrHvGVJSVcJSyqirlraZWi8CPr4EFZ8DHfgveGWzJGgrCjkfh9Qegf5/5h116Kax+N7SsALfXrC9//jtm1Uc6CekUtK2Ci/4KKmbuogThuCe4C5ZfATXz4MCrczuWrpfgV7dC+7nwrm9DMgrxEBzcBN0bIdQL3S/Blgend9yPPQZVF87OmC3KSjh0XV8LrF2zZs0nZ3qsXfOvYf2efm654nR8T/4DPHQrrL4BUnFIRECdDY1LINwP4T7Y8Vt46ftQ2QAd50JVC1Q2gpGCN38P+9eDkYbGpbDwfBg5CM/9Jzz77+D2wUlrwO03haVhCXgrAQ12/hZe+YkpMo1LwBuAdALCA+bVE4Z5JXLodVN81Fnm1cqCM+DENeAu8k88csj8AC+73LySmYhkzBS5g5vh1f+BcNB8n0bafO/ObSNzu7LBHKc3YH5ZkjHz/1XNcN6t5vsQhFITG4HRw9B4gvnZ3PorWPev1pW8dTWvuUBzQ+18qFmQuR8t9zkud+Z16ZT5HTBSmdvpJMRGze4TRlZHJXvRYf9eePobULsAPvBTqG7NPGfxW3LHnYxbx8/6XtnnSSUyF5nppDk3NBS20+lMKCvhKCXhmqV8M/k+bjznCnxGHJ76F/ODNhnLrjD//+bvzQk0nTR/bl0Fb/0bOOmd5oRuf3hGDsGedXDgFXjtZxAbhov/Fi79YuY5+ivwzDeh83lTVGy8AfMD4nKZH6yOc80JeP8LsOV/zQ9XXYf5wUwlzH+JkDmmQLP5wU8lLIGrN+2u22teiTUtg8e/BKEeaD4JllwM/mrzdqAp8wHd+TvY/HPzNoC/DhoWZX0Rx3zB7J/7dpruKxu3zxTlvp1w7bfNL5srKwWXiEJ00PzipLMEKRExx928vLg/tHD8YFdUNS0zP+eu/ze3CwFPeidcd+fUYXCPD/AdkSEVigjHBHg95qSVSBlwyefgtA9AdMic4Fxu6HwORnvNkFOgyZxUW7P2JzcM8wonFTevpPNRMw9Oe7/579IvQu9O6Dgn9znqTPOKBMy682TEHMNkV+WppOlUXv0fSITBX2MKgy9gxkBDfeak63KbV2DBXZmrl1BPxhld+gV4+cew5Zfme7EFwsZTCWfeCPWLzCumVdeBr6qwX3B0yDyPp8J0Wi4XPPFV8ypsx2OmTdfcpkC6PRAZmPx4iy6CU94Lde3W7yBh/u5t8XZ7TVd2eFvm6tAWofZz4OLPziwUWSiGMV4UZ4NkDHp3mL/ndCIrLu4xf68uV9Ztj/k7rmwAX435e4kNQ2TQFOtUYurzZZNOmJ+X2AhEh81jpRKZq3U0s9Wpc1sr7HYyan4uUsnMhZXtBjAyV/aGgdM31eWxvi++TEVV0zIzBPzlw+MdMYY51mHdvLAj65iOa8520mnrd+jO5Bnsn33V4K/N39rE5Ta/N8do25OyEo5S5ji8LvMPmkhZi4QaFuU+YaorXE2bXl6iom68aIzF5SpsYnZ7YOU15r/pEhkwXU772eaYzv64eX8qYV6xxUetL4kX6pQ52RRDRd34+y79khmi691p/r7TKdNRJKNQ3WYKsONg3Obv2FNhhv2evwN+81eTn1NzQ/OJ4PFnHJCRgmf+DbY9ZIYwUvHcL7+WNcE7bieVFR7IDlMkzfGimcLrqzIn0ZTl0OIh6NlmTsj2hGIYmdBDdiivTpnHSVtuMTJoXjTYQuv2ZCZIZ+LLCgeGg+OFvhywRcB577ZIaFkhJVtMsC6I4ubvGKB6nvm3ASsEO0EYtrIe2k6ezXdyTFNWwlHKHIfXbTuO42x1aWUDLHv7+Pvd3lxHNRtomhnSK4bzbs26StTM8bq9mUkmGYWGxfnF/M0/mCGL0UPm8x1xSGcmHPvKd+yVustthhLs+7wV5uv795oTvR0CdHnMSX/VdVDVajoBOz9lOwBbzEJ9MHzAvG2/trLedHipmOkmsq/gnatyOwavmS54/mlWWNKTEbbUmFi8fTuVgEi/GZd3uUxhq2yAivrp55xcbvP1/hrzAsFfM2ayT0/jNpnbHt/EV/BTYf8t7XCpMCPKSjhKSU6oSjj6cbnNooD6hdN/7fLLzX/C7OJM+HMwcbtclNmytTlFfpMTMC5UJQiCIAAiHBNy3IaqBEEQpqCsQlUlTY5LqEoQBCEvZeU4StmrSkJVgiAI+Skr4SglGcchwiEIgpCNCMcE2DmOpISqBEEQchDhmACPFaqKi+MQhLIjHE9y97N7MQy5MCwGEY4J8EmoShDKliff6OWffrONN3tG53ooxyQiHBMgoSpBKF8iCbMjgFwYFoeU406AhKoEoXyJWsKRlq93UZSV4yhlOa6EqgShfLGFIyU5jqIoK+EoJRKqEoTyJZY0LwhTafl+F4MIxwR43LIAUBDKlZjtOEQ4ikKEYwJ8luOQHIcglB/iOGaGCMcESKhKEMoXJzkuOY6iEOGYALdLQ9MkVCUI5Ug0IY5jJohwTILX7ZLuuIJQhsSSUlU1E0Q4JsHndonjEIQyxHYcaXEcRXFMLABUSl0PXA20Anfquv67I3Fer1sT4RCEMiSalKqqmTDrwqGUuhu4BujRdX111v1XAd/C3ID4h7quf32iY+i6/hDwkFKqAfg34IgIh0dCVYJQlsRsxyGhqqI4Eo7jHuAO4Cf2HUopN3AncAXQDWxQSj2MKSK3j3n9x3Vd77Fuf9l63RFBQlWCUJ7YjiMpjqMoZl04dF1/Wim1eMzd5wK7dF3fA6CUuh+4Ttf12zHdSQ5KKQ34OvCYruuv5DuPUupm4GbrdknGLqEqQShPYlJVNSPmKjmugK6sn7ut+ybiL4DLgfcopW7N9wRd1+/Sdf1sXdfPbmpqKskgPW6XrOMQhDLEdhwSqiqOuUqOa3num/AvqOv6t4FvT3XQUnbHBbMcV1aOC0L5kXEcczyQY5S5chzdQEfWz+3AgZketJTdcQF8EqoShLLEXsch5bjFMVeOYwOwXCm1BNCBDwAfmulBS+04JFQlCOWJs3JcQlVFMeuOQyl1H7AeOEkp1a2U+oSu60ngNuBxYDvwgK7rW2d6rlI7Dq9bk1CVIJQhMVnHMSOORFXVBye4/1Hg0VKeazZyHKOxZEmOJQjC0UEqbTjrsyQ5Xhxl1XKk9I5DQlWCUG7YnXFBul8XS1kJR6mRdRyCUH7Ye3GAOI5iOSZ6VRWKlOMKgjAV2Y5DchzFUVaOQ0JVgiBMRY5wiOMoirISjlIjoSpBKD9yQlXiOIpCQlWT4JUmh4JQduSGquZwIMcwZeU4ZiNUJW3VBaG8yHYcEqoqjrISjlIjoSpBKD+yHYeEqopDQlWTIKEqQSg/7HYjII6jWMrKcZQ6VGXvAGjIh0sQyga73QhIOW6xlJVwlBqf2+z+LruECUL5EMt2HPLdLgoRjknwus1fj4SrBKF8iIrjmDEF5TiUUp8GfgSMAD8EzgA+r+v672ZxbHOOxxEO+XAJQrlgOw6f2yUtR4qkUMfxcV3Xh4ErgRbgY5h7gB9VKKWuVUrdNTw8XJLj2aEqcRyCUD7YVVWVPrc4jiIpVDjsrV7fCfxI1/VN5N/+dU6ZjXUcIMIhCOVELJnGpYHfI46jWAoVjpeVUr/DFI7HlVI1QNnPpnaoSvpVCUL5EE2kqPC6cbs0cRxFUqhwfAL4PHCOruthwIsZriprvFaoSjrkCkL5EE2awuHSNGk5UiSFCscFwA5d1weVUh8BvgwMzd6wjg58EqoShLIjlkjj97gsxyHf7WIoVDi+C4SVUqcBfwt0Aj+ZtVEdJUioShDKj2gynQlVyVe7KAoVjqSu6wZwHfAtXde/BdTM3rCODo6WUNVzu/rE9QhCiYglUo7jkF5VxVGocIwopb4A3Ag8opRyY+Y5jipKX45rhaqSczdpv3FomA//8EX+uP3wnI1BEMqJaDKN3+vGrUlyvFgKFY73AzHM9RyHAAV8Y9ZGVSQlL8f1WKGqOfxw7Tg0AkB/KDFnYxCEciKaSFHhceFyadLksEgKEg5LLH4K1CmlrgGiuq6Xf47DNfehqt29IQBCseScjUEQyomY7Thc0la9WAoSDqXU+4CXgPcC7wNeVEq9ZzYHdjTgPQpCVXt6RwEYEeEQhJIQsxyHWxPHUSyF7sfxJcw1HD0ASqkW4A/Ag7M1sKMB31EQqtojjkMQSoq9ANAlCwCLptAch8sWDYvgNF57zGKHquaqoimdNtjTZzqO0agIhyCUglgyjc92HCIcRVGo4/itUupx4D7r5/cDj87OkI4e7FBVfI5CVQeHo85uZaPiOAShJCRSBl63SxzHDChIOHRd/6xS6gbgIszmhnfpuv6rWR3ZUcBch6rs/IamiXAIQqlIptN43RpuTSMpK8eLouA9x3Vd/yXwy1kcy1HHXIeq7PzGspZqEQ5BKBGplIHbpeFxa8SS4jiKYVLhUEqNAPl+sxpg6LpemgUTk49hJfBpoBn4o67r353tc9rY6zjmKlS1u3eUar+HJc1V7O8Pz8kYBKHcSKTTZqhKk5YjxTKpcOi6PqO2Ikqpu4FrgB5d11dn3X8V8C3ADfxQ1/UJN4XSdX07cKtSygX8YCbjmS72yvG5ClXt7w+zqClAdYWHEUmOC0JJSKVNxyEtR4qn4FBVkdwD3EFWQ0SrXcmdwBVAN7BBKfUwpojcPub1H9d1vUcp9S7Mtu53zPJ4c3BCVXPkOEKxJDUVHmr8HkJxEQ5BmCmGYZjJcZdmtVUX4SiGWRUOXdefVkotHnP3ucAuXdf3ACil7geu03X9dkx3ku84DwMPK6UeAX6W7zlKqZuBm63bJRm/26WhaXOX44gm0jRX+6iu8DAaTWIYBpp21G28KAjHDLZOeNwuc+W4LAAsitl2HPlQQFfWz93AeRM+WalLgHcDfiYpAdZ1/S7gLoA1a9aU5NOgaRpet4vEHF2V2AuVqvwekmmDmNUOWhCE4rAvAu1Q1Vwu7j2WmQvhyHfJPOFfT9f1p4CnCjmwUupa4NrFixcXM668+NyuOUuO2zuV1fjNP9NoLCnCIQgzwA5Ned1mqEpyHMUxF6u/u4GOrJ/bgQOlOHCpu+MCNFb56B2Jlex40yEST1PhdVFlCYe0HRGEmWFvyuZ2WTsASqiqKObCcWwAliullgA68AHgQ6U48Gw4jvaGSroG5qYU1txwxk21JRxSWSUIM8Ne8GcvAJTkeHHMquNQSt0HrAdOUkp1K6U+oet6ErgNeBzYDjyg6/rWUpxvNhxHR0OArv5IyY43HexQVXVWqEoQhOKxcxpSjjszZruq6oMT3P8os9DrajYcR0djJX2jMSdRfaRIpc2ywUqvm+oKCVUJQimwhcMroaoZUVYdbmfDcbQ3BADoPsLhqmgiBZCT4xDHIQgzI5lVVWU2OZzjAR2jlJVwzAYdjZUARzxclRGO3KoqQRCKJ2Elxz1WjkPWcRTHXCTHZ41ZCVXNleOwSoBzHIckxwVhRtjJcI8VqkqK5SiKsnIcsxGqaq724/O46Bo4so4jEs84joDPjaZJjkMQZoq9ANBjr+MQw1EUZSUcs4HLpZkluUe4O212qErTNKp9nlnZd3yLPsQWfajkxxWEo5HsBYBuF1KOWyQSqiqA9oYA3UfYccSSGeEAqK7wzIrj+Ooj2wC4/+YLSn5sQTjasNdxuF3WDoCS4yiKsnIcsxGqAuiYg0WA9paxFdaeIFV+z6wkx8PxlCTdheMGe+W412Ulx8VxFEVZCcds0dEYYDCcYCSaOGLnzA5VAVT7Z2dPjlgi7eRTBKHcyV4A6BHHUTQiHAXQ3mCW5B7JcJXjOLKEYzZCVbFkyjmXIJQ7tnB43GaoyjDMPTqE6VFWwqGUulYpddfw8HBJj2uX5B7JBHnEchyVWcIxGyGlaCJNWDaJEo4T7PJbjxWqAkmQF0NZCces5Tga7bUcxTmOw8NRekai03pN9spxsHIcsxGqSqYckRKEcifjOMyV44CEq4qgrIRjtmgIeAn43EUnyP/yvlf5zC82T+s1tnD4LcdRV+llKFL6HEssmSaaSEuSUDgusJPj9gJAEMdRDGVVjjtbaJo2oy65e/tC0/5wxrJWjoMpXqF4ingyjc9TOr23zxNNpgj45OMglDd2Oa7dcgREOIpBHEeBdDRWFtV2JJ5M0zsaIxiKExwtfEOoaCKFppk7EAI0VPkAGAzHpz2GiUik0s6XRiqrhOOBjOPIhKrSUhsybcpTPKQHAAAgAElEQVRKOGYrOQ6ZRYDTrcDoGYliv2Tn4dGCXxdNpKjwmKvGARoCpnD0l1A4Yllb4kqeQzgeyDgOF25rE2vJcUyfshKO2UqOg1mSOxpLMhieXp7h4FAmKf5mz0jBr4sm0lT6Mvt/NAS8AAyESpfniGWJhTgO4XjASY5bGzmBhKqKoayEYzbJ7MsxvTzHgcHM89+chuOIJFLOqnGA+kDpQ1XiOITjjbyhKnEc00aEo0DsfTk6+0O83DlQcMjKdhwntlWz8/B0HEfujoONVo5jYJqOZ6pz2Ix1HL9+Tefqbz8ji6OEsiJ7AaBHHEfRiHAUiO04bn/0DW747vM8u6uvoNcdHIxQ4/dw5sIG3uyZTo4j7ZTiAtTboapZchzhMY5jiz7E1gPD4kSEsiJ7AaBLqqqKRoSjQOoqvdRWeNCt0NNzu4IFve7gUJT59RUsb6uhfxqVVbFkyinFBbP1SKXXzUBodoQjOsZxhKyfhyPHxqryz/5iE//62zfmehjCUU72AkDJcRSPFO5PgzMWNmAAw5EEL+yZhnDUVbK8tRqAHYdGuHCZf8rXRRMpp92ITUPAW9JQVU5yfIyzsPtiDUcTzKurmPax9/SO0hDwOWXEs81rXYO01U5/nMLxRd4FgBKOnTZl5ThmsxwX4Ec3ncM9N53DW5Y187o+VFDvqINDERbUV3BaRz0ujYIFJ5pI5+Q4wFzLMWuhqrGOI2Y7juKE6mP3bOBbf3yz+MFNk2gyJT23hClJpdNomtkd1w5VSdeE6VNWwjGb5bhg7gbocmmcv7SJVNpg477+nMdf6xqkPyuUFEum6BuNM6+2krpKL6e21/NMgbkRMzme++dpCJRWOLKT49EJHEexbU76R+PT7s81E8xmjZKPESYnkTacpLg4juIpK+E4Upy5qB6vW+OFPRnh+PVrOtff+Rxv+8aT3Lt+HwCHh8x8xvx6M4Ty1uXNbOoaLGgyjlgLALOpD3invY5kMnLKccdMuvbV+3CRe5BEEqlZ2T9kIqJxadYoTE0qbeBxmdOeJMeLR4SjCAI+D6e11/P41kNEEynW7w7y2V9s5uxFDZzWXs/f/XorT77Rw4EhM5G+oM4s5X3r8hbSBqzfPXW4amxVFZiOo3+WkuNjJ107DFdMcjyRSpNMG0WHuYrBDFWJcAiTk0ilxzkOaTkyfSQ5XiS3XbaMm360gU/99BVe2BNkUVOA//7oOVT63Kz51tP8/cNbaKn243FpLLMS42csrKfK5+bZXb1ctXrepMeP5QtVVfkYjiZIpQ3nQz8T7H3NYXyOIxwvPsdhv/ZIOY5U2iCRMsZVhgnCWFJpA4/bFg7rPglVTRtxHEVyyUmt3HzxUv74Rg8tNX5++mfnURfw4vO4+OfrV9PVH2Fz9xB3fOhMpyrJ63ZxwQnNPPr6oZwV5fmIJvNXVRlG8XmHcedIZDrwjs1xjM4gx2Efa/gICYd9vnAiJQsWhUlJpAzcVqjK/r+EqqaPCMcM+MyVJ/GVa1dx383n05pVCnrhCc187U9Wc8/Hzh3nLP72qpNIJNN8/J4NeauyBsNxRqIJEiljfFVVwF49nhuuKjYPYTuO+kpfTqjKMIyM4yji2JEZvLYYbOFIpQ3iKYk7CBOTTKXx2o5DchxFI8IxA3weFx+7aAnzrRxGNh8+bxFvWd487v4T22r4r4+cyY7DI9z19J6cxwzD4D3fW89nfrEJIG+oCshZBPjH7Yc5+5//wMGh6e8VErMcR12lNydUFUtm2q2PzXEMRRJ8/pebGZlEFGwRiifTOeGw2SI6SZJfELLJDlW5XJn7hOlxTAiHUqpKKfWyUuqauR5LKXjr8hYuXt7Czzfsd1ogALzZM8qunlHW7ewFyOM47LYjmUn7pb39xFNpXu8emvY4Ysk0PreLgN+dE6oKZTmhsa7hpb393L+hi1f3D0543Gz3ciTyHNliIQlyYTISWVVVtuOQJofTZ1aFQyl1t1KqRym1Zcz9VymldiildimlPl/AoT4HPDA7o5wbPnTeQg4Px3hyR69z3x+2Hwaycg+eqUNV2w6aix13HCq8gaJNLJnC73ER8LlzJl978R+MFw67Zcpk60myj3UkhCM6yQp4QcgmlR5fVSWOY/rMdlXVPcAdwE/sO5RSbuBO4AqgG9iglHoYcAO3j3n9x4FTgW1AWfWTuGxFK601fu54chcHhyJcsaqNJ7b30FTlI2iFoip8ucJhd8g9bHXcNQyDbQcs4ZhG510bu+TX7IGVEYiQtYajwusalxy3xzZZWXC2cByJktzscJiEqoTJMJPjdqhKFgAWy6wKh67rTyulFo+5+1xgl67rewCUUvcD1+m6fjswLhSllLoUqAJWARGl1KO6rh/zGVCv28VHL1zMNx7fwaauQb79x130h2Lcduky7n2hk4FwImc/DoAqv4cV82p4YW+Qv2A5vSMxZyKfqmV7vhJe23FUeHNDVfbiv/l1leMEos92HJMJxxEOVdkODSRUJUxOKm3gdY8JVYnjmDZzkeNQQFfWz93WfXnRdf1Luq7/FfAz4AcTiYZS6mal1Eal1MZgsLB+UHPNpy5dxqavXMna296Cx6WRNuDtK9s4e3EjMD7HAfCWZc1s2DdANJFywlRnLWpgT2+IeDK/nm7qGuTcr/3BWdFuE0um8XutUFXWZD9qharm11UwEk3kfLGCo5bjmCxUlSMcs+848omeIOQjkUo7F1ASqiqeuRCOfCvXpvzL6bp+j67rv5nk8buAfwRe8Xq9MxjekaWu0ssp7XX86lMX8q0PnM6p7XWcs7gByC8cFy1vJp5M89Lefkc4rj99Acm0wd6+0Ljnb+4e5MM/fJFgKM6Le3N7a8USafweM1SVfaUejmUcR9rIhK4AgiHbcUwsCNE5dBwSqhImw3QcVqhKkuNFMxfC0Q10ZP3cDhwoxYFnu8nhbDK/rpLrTldomsaa1fM5b0mj04o9m/OWNOJzu3h2Vx/bD46g6is5Z4npUPLlOe55bh8et8bpHfXjtq619/yoGOc4zMl+gdVjK3shn+M4JglVZYvQkVjLESlhcjydNvintdvY3Vv4plvCsUMyK8dhl+UmxXFMm7kQjg3AcqXUEqWUD/gA8HApDjzbbdWPFB2NAX5+ywV597II+Dycuaie32w6wPrdfaycX8vS5mo8Lo2deSqrXteHOGthA+cvbWJP3yiJrPJf03G4CHg9xLPWbtgTv73ifSir/NfOqRRSVaVps7N6fCic4MWs9vS5oaqZCceh4Sh3P7eX3287PKPjCEcnyXTayXFIk8Pime1y3PuA9cBJSqlupdQndF1PArcBjwPbgQd0Xd9aivMdy45jOrzj5HkcGIpS6XPz3rPb8XlcLG2pYsuB3LUcoViSXb2jrFZ1nDSvmkTKYF9WOMtMjrup9JkfA/tq3XEc1sJG2zWk04bjNCZzHNGEmXSv9nlmJcfxw2f38OEfvugIxmR7p08X+33lqwb7p7Xb+PHz+2Z0/GyiiRQ9w0eu9bxguouxOQ4JVU2f2a6q+uAE9z8KPDqb5y5nbrpwMTec1U5tRSaXc+EJzdz30n7C8SQBn/ln3XZwGMOAU1Sd4x52HB5heVsNYCXHPS6nJ1YknqLa7yEcT+J2abTUmDsV2pPoUMRssFjpdTMQjmMYBpo2PmUVSaSo9LkJeN0F5zhGogle3NPP5avapnzu1gPDJNMGvSMxOhoDk25INV1s4RhbhmwYBg9s7OLsxQ189MLFMzqHzV1P7+En6/ex4UuX5/09CqUnmRq/APBo7lIzEIpz0z0b+I/3ncbSlvGh67nimFg5XijlEqqaCk3TckQD4MpVbcSSaZ59M7NRlL2a/JT2Opa1VuPSYGdWnsOsqnI7SXj7yj0USxHwuamrNM9hh5vsxPgJrVUkUsaEOyBG4ikCXjc1Fd6C13H86lWdP/vJRjqDuQn+wXCcy/99HVv0jJuyFzv2WqXB0UQKTTNbwIQTMwuN2SG4scJxaDjKaCxZ8PvpG41N6X46g2H6RuMFid1rXYN89Tfb+P663SVtrX+8YYaqcluOTFaO+/XH3ih4187ZYLM+xKauQTZ1T9ypYS4oK+E4XkJV+ThnSSO1FZ6c2PwWfYiWGj9ttRVUeN0saqrijYPDfOXXW1i76YC5y6DH5TgUewILxZJU+z2OONmTpZ0YX95qOpaJKqvCiRQVPjc1FZ6CHcdhK2RjL2i02dMXYlfPKL/dcggwnYludRbuHckIR4XHPW4F/ET8xX2vcueTu/I+NpHj2NUzap1/6vczGkvyjv94mn99/I1Jn9dvCbH9ewX4x7Vbue+l/eOe+/11u/nhs3u5/bE3eGBj17jHhcLITo5PtQNgLJnie+t28/jWQ0dsfGPpHggDR26LgkIpK+E4XhxHPrxuF5euaOWPb/Rw7wud/M8LnbzWPcipqs55zvLWan6//TA/Xt/JQ6/qzjqOsTmOUDxJwOemusIUFHsStRPj9v4iE63liMbNlvC1lV5GYgVeoY+Yx9p+MPdvZ597g7VNb/ZCxx5LOCLW3iWV3qmFI5U2eHzLIb7zxJuO8GTj5DjGfFHtirRCqsR++kInwVB8yv5h9rlsJwewdtNB/ri9Z9xz9/SGuHxlKzV+D4eGJC9SLMk8CwAnSo73DJt/l3CssPDnAxu6imo2OhndA+bxRDhmkePZcQBcvrKN/lCcv3toC19+aAt7ekOszhKOE9tqMAxwaaAPRoglzOR4RVaOA8xQVbXfg9ul0RDwOm7A7lNlC8dElVWRhCkc03Ec9uS57WBuZZjtdjZ1DxJPpnkjq3Is4zjSVHrdVPrchKcoxz00HCWeShNNpPnBM3us16f4u4e2EByNTZgcf7NAxxFNpJzj7jw8Mun+IGPbtxiGwVAkzlAk9/eaThvsDYZY2lJNS40/r+AJhZHMWgDomiI5ftAS6FABi0qHown+9pebeXBjd4lGaqJbwnGktigoFNkBsIxYs3oe/3rDqZzWUU8wFOO+l7q47vQFzuM3nNWOgUF/KMFvNh1wkuN2qCpi5QdCsUyC/dT2el7ZPwBAnxVSOcFK0k3UdiSSMIWnpsJTcE6gdzS/47D3WI8m0mw9MMTOQyNU+UyxywlVeQsLVXVaVWULGwPcu76TP7/kBN44NMK9L3Ryekf9hDmO3ZZwhOMpkqk0Hnf+a65fbOyibzTO1afM55HXD9IzEqOtNn+bNcdxWO89FE+RSBnj9pXXByPEk2mWNFfRUuOnZ0QcR7EksxYA2o4jmcovHIesC6ZCclB2yXqpJ3gJVQmzjsft4n3ndHDSvBouPKGZ73zwjJxKjCXNVXz2HStY0hxgJJYkmTbwe9y0N1SiabBFNyftUDxFld8UjrMXNbDz8ChDkQTBUIyGgNeptuodifGDp/eME5BI3JzIayq8jESTBe3KZ7sZfTDCUCThJOqzJ/CXOwd449AIJ86rybnytps1BryeKVuOdPabX8Q/v+QEIokUbxwacRyVPhjJyXFkj/vNnhHsVl8TfYkNw+B/XtjPKaqOD5+/0Hzd4fwLCaOJzB7pztoY6/+DY0TL7giwtLmK1toKJ0R3vLI/GObnG8bngQohpxzXPbnjOGSFnUITFIFkY39OSz3BS6jqCHA85zimg6oPOLcrvC6aq/2c3lHvtHUPxZJU+c3w1VmLzPYnr+wfIDgap6naT22FGcb6+cYuvvbodn71qp5z/EjCrMqqqfCQTBs5LUHyYRgGfaMxZ6X81x/bzslfeZx9fSGGIgmqfG4WNgZ4bMshth8cZoUtHJbY2CvgKwtxHMEwPrfLeV+HhqKOcHQPhB3hSKUNQvbEPhpjIJzgpHlmCHQkmuRrj2zjr3/+Ws6xX+4cYMfhET5y/kKngGCi5pPBLLG1k+T25DMUzhWtPdYq9iUtVbTW+OkZjh3XW+T+fON+PvfL1yfszTYZyVQ6TznuRMJh5TgKcRyzIBzRRMq5SDgSPd+mQ1kJx/Ge4ygU1ZDZsdBvdeC9fGUbm7uHODwcJRxPOo7j9IX1uF0aL+8bIBiK01jlQ9M0GgI+9vSaV8LbxoSXInZy3K7KmuJDH46niCbSzo6J973URSptsKfPdDp1lV4uWNrEy50DDEeTXHBCMy01fvrGVFVVet1TthzpDIZob6xkQb35Ozg4FOWwlQTtHojQH0o4oQw7zGbnN85aVO+8n42dAzy1IzeJ/dMX91Pj93DtaQtorvbREPA6rx1Lf1YllR2qssNk8VQ6533s7QtR4/fQUu2ntcZPJJGasBS6ULYeGOJddzx7TJb29lvVfMUs9kymjfH7cUwgwvYFRSE5Dls4ShmqOjCYSbSL4xDmHFWfJRxWYvzylebCuz9u72E0lqTK2gsk4POwan4tazcf4LWuQZY0VQHQWJVZRzK2hDZi5RzsdSBTTU52q/ZV82tprPI5YtY3GmcwnKC20suXr1nJw7ddxKavXMm7TlvghKoMw3CqqgI+s1nj77YeylvSCqbjWNQYoMrvobbCw8GhiBPL7h6IMBCO09FoOjJ7MrDXjZy50HQpw9EEwdE4A+GE895iyRSPvH6Q689QBHweNE1jeWsNb07oOMz37NKy27hkJp3sPMeevhBLWqrQNI3WWjNMONNw1T3P7WNz9xAv7T02OklnM2gJ7GgRnZCTacPJT7mmaKtuV0gVUlWVEY7STfB2mKqxyieOYzaRUFVhNFdnJmf7/ye2VdPRWMmDL3cRTaQdxwFmuKozGGZebQWfW7MCMHcjdLs0/uQMxZs9Izlhg6i1cnzFPDNck714Lx920r25xs/X330Kd990jnV/jGHLcdRUeDm1vd4Ro9aaCuKpNMORpFlV5XM7oarvrtvNd5/aPe48hmHQGQyxyBK/+XWVHByKOm0/9veHSaUNRxztyeC1rkFaa/ycaK24H4kmnZyMHUY6OBglnkxzWke9c77lbdW82TOaN6xkC86ipqpMXiWrSi1HOHpDLG2uct43ZEpFiyEUS/LI6wcBs5dZPl7vHuI7f3yz6HNMxN6+EJu6ZraYzXZm4SJclxmqGttWPf9zbSc6HcdRygneFo6V82vEccwmEqoqDE3TnHCVXYqraRofvWAxr1h7iVdnCcc7Tp7H0pYq/vujZzu7EF5z2gL+/JITuHRFK4mU4SyQS6TSJFJmW5ITWqqp8Xt4LWui2Nw9OK6BoO04Wqr9XHnyPC5a1kyVz03fSJyhSIL6wPg2+U6CfjSaswAwFE+y49BI3pLVYChOKJ5iUZPpKObVVXBoKMqh4SjZHT8WN+cKxyv7BzhjYb0TeusZjjr5DztcZy9KzHZzy1urGYokJl0vsqy1Oms73izHYZXkRhMpDgxFWNJs5n9aa2zHUXxl1W+3HCIcN/NQr+v5L7Ie2NjFN3+/s+RVQv/2+A7+8v5XZ3QMW1RD0wxVpdMGaSPTFdcudrBDVd99ajef+tkrznPtUFUknpoypzTTHMeunlH+4eGtOe6neyCMx6WxrKVahEM4OrAnOH/WLoN/9talrPvsJXzl2lVcd3pmb60LTmjiib+5xOlxBXDj+Yv4mytPYtV8U6TtPIcdmw/43LhcGqd11PPq/oxwfOPxHXzqp6/kLGKzY/xN1ZluwM01fvpGY06OYywt1ZmQjbMFrs9DNJEmHE8RSaTGVcPY7Uxs4ZhfV8HBoQiHh2OclPXebOEYjiQIjsboDIY5c2EDtZWmmO7tCzvPtduv2/X27Vn5o4XWeboHxy8KC4bieN0ai5sCBENm36+BMY4jmkjxj2u3YRhwkuXebMcxk7Uc//tqN4uaAlx9yny26kN5J0W7DHRfnj1eZkLfaIzugUhOl+bpUqzjsNun245D0zRcWiZUtXFfP+t3m6G7vlCMZNpgXm0FybRBfIrxOqGqIrdKfvg1nXue3+cUfIDpOObXV1AX8DEaSx5VXXxFOI5T7AnO78ndLGpRUxUfu2iJc0U/FUuaq6j0up08R9S6CrSdzOkd9ew4POJctW09MEw8leZ76zKhJNtxNFVlztlcPYVwZJUEx7JWjmczdnLtDIad9whmqKpvNE48meZMq8oKyAlV2aJ3xsIGx4Xt7cskvHdbjqN7MIJLy7Sih8nDSv2jcRoCPpqq/cSSptgNhhP4LCEfDCf48kNbuO+l/dxy8VKusJo/1lZ68HlcRec4UmmDjfsGePuKNk5pryMYijsL3bKxwyT5NgeziSVT/P2vt0xrJbvdKNMW2rG83Nk/qViZAluc47An3uw1OG6X5jiOoUiCgXCcVNrgsFVRtbTF/CxMleewhSOWTBdV7bXX+mxml58fHIowv66SWquDg10QMRpL8sahuQ3Hi3AcpziOwzuzj4DbpbFifg3bDpqxcttx2JP4GQvrSaUNXteHODQcpT8Up7bCw30v7XfCLcHRGLUVHmfSBDMPc2AwQiSRmlI4osnMAsBssq/ewFyn4ve46GjIOA4bO/ENpiPRNPPq8dWuATwujVNUHR63iyqfm33Wl3xBXYWT49AHIrTVVjjtLICsRPb4idWuULNDf8HROIPhOIusxPxgJM7LnQO84+Q2vvDOlU48XtM0qyS3uFCVPhAhlkxzYls1Jy8wuwqMzXMYhuEIx74sdzWW17uH+Mn6Tn63rfBeTvbEuL8//3H/8r7X+KffbJvw9ZFEypmYsx3l5u7BKTffSqTN19mOA8wEuS0og5EEhmGO0U6M24tdp8pzZDuNYvIctlhmH2cgnKCpykeNJRz2cX/07F6uv/O5Gbm2mVJWwiHJ8cJptybPCs/47Wmnyymqjs3dQ4xEExnh8GUcB8Cr+wccV/L3155MIpV29rboG43TPMbhNFf76bImr3zCYQvNwaGok1Oxz2mX0/aNuSp/cW+QMxbWOwKV7Q6WNAdosibxpmofNX6P4zhWzq91jl1T4XUmvXOXNLK/P0wilUYfDOfkN8B0UG6XluM4tuhDPLmjh/5QjKZqH81WeC4YMteKzKurwO9xERyN09UfzttKu7XGX7DjeOKNw3w1ayLe1WtWeS1vq2bV/FpcGmwdIxzBUNz5O+4LTnz1bwvoZOIyFls4OvMIRzKV5tBwlI37+iesdMrOA2VP5v/3gU3c/ujkTSVTqdxQlX3bEQ7r2P2hmJPfcBzHFO4m2ylMt7LKMDL75GTnlAbDceoDZmEIZPInnf1hool0TnPMI01ZCYckxwvnilVtfGHNClYtmPnv6oYz2wnHUzz4crdTW287jqZqPwsbA2zYN8DWA8NoGly1eh5vX9nGfS91EU2k6BuN0Vw1XjjsL3RtHuHQNI2OhkqnVNYux4WMe8h2HEORBNsODnPekibnvvljwkrtDZXO/iR1AS99oTivdQ1y5sJMpVRtpccZ1zlLGkmmDTqDYfTBSM76GDDdWHO1z5mEAP7j9zu59d6X6QyGaazy02i97/5Q3JoofNQHvGw/aO45srgpwFhaawpfPf6T9Z3893N7iSXNv4u9kn1ZSw2VPjfLWqvZMqac2nYbbpfGnknCRvZkt7+/sDxI3ArJAezPI0h9o2aYaDianNA9ZHcpsMNH6bTB/mCY3ikKBhzHkeUKXZZw2H3CwHR/h4ajeFwaCy0HONXq8eFIwvn8Tddx9I3GGbGObwuQYZitZ+oD2Y7DfI79eZrLnmVlJRxC4VT5PdzythOcEMhMOK2jnjMX1vPj5/cRiuXmOADecXIbT+7o4Yk3eljcVEW138NNFy6mPxRn7aYD9I3GchLjQI4DqQ+M30IXzPbudpilwut2xOqiZc24tFzHsXFfP4YB5y/NEo4sh9Ba66ejMUBLjd/Z7+Tpnb2E4yneurzFeZ599Vflczuhnh2HRjg4GB3nOADaxrQI2d8fJpZMEwzFaaryOS4nGIozGEnQEPBSX+lzOusubKwad8zW2kyoKpFK51StZZNOG7zSOYBhQFe/KQa7ekZprvZTZ1WqndhWw5s9uWtNuiw3cEZH/bh8Q89IlC/872aGown2WpO/7TymIvuqPF+oKruz7MbOgSmPYTuO3tEY8VTaKeueiGQex+F2aaQNg7DVJwzM5PvBoSitNX7n7x3O6iLw+V9uHrcAcyiScEKg+SqgBkJxbvvZK04FXTbZrm44ksljJNMGDTmOw3zvdk6pdzQjlJ/62Sv85x92Tvr+S4kIh1ASPnbREvYFwzy6xVwfkJ1v+NMLFmMYBq91DToO58ITmljeWs03Ht9B10CE5upcx9GSJST5QlVgrj2xJ5IKj9sRmFPb62is8uc4jhf2BPG5XZyR5R6q/R5q/B5r0aGbz1x5Et/6wOnOOUeiSXweFxcuy4iNffXXVO1n5fwaavwe7t+wn2TaGOc4wAwr2VeI2bkDwBQO6332jsSs0mMfdQGvcwW6uHm845hXV8FwNEkoluTh1w5w/Z3P8er+8RPtrt5RJ2xiV5Tt6h11WruAWQ7cPRDJ2X7XHuNbl7eYCeOsq/z/enI3973UxZNv9GQcRzCct+Jnw75+zv7q753ih+xJvzOP2GQn2Tfuyy8c2ZVntguwhS4YmrwVS97kuJXjyB5bMBTn8HCUtroK53Nsn+uh1w5w/4aunIWThmG6JLvgJF9l1eNbD/GbzQfZ2DlAOm3wjcffcCrXsgsQ7NfaYbPJHIcdAn29e4hHNh9k3c7eCd97qRHhEErCVavnMa+2ggdfNttKV2YJR0djgCtXzQPgZEs4NE3jc1etYF5dBctaqrlsRWvO8ZqyhGQi4ViWVULr97o4c2E9P/n4ubztxBaaq305Vv7Fvf2c3lGf44TAnITttRGLm6s4a1FjzjnPX9rkdAoGnLUcTdWm2Fyxqo1nrF0X8zmO1toKZxx27uB9Z7fj0qC9sZKAzxSuDZYjqq/0Um+d2+dx0VYzvrPuYqvqa18w5PTCeiBPO++Xs67a9wXDGIa53mZZlnCc0FKNYZjrUZ7a0cOPnttL90CY+oCX1cr8W/1+22H+32/fYE/vKF3MNQ0AABoESURBVPdbzQVf7hygMximyucmbuUmwMxlfeXXW0inDZ7e2UvfaGZfEjsUtLSliv394XGTvF3ddfaiBl7u7OfFPcFx1UN2jsOlZaqqbKGzS7Enwk4m5yTHLceRveBywKo0m19X4SyEtY9rT87ZeR27VNYWjnyOY721i2B/KM6+YIg7n9zNt60Flnv7QnhcGn6PyxEwWyAbArnJ8Ug85VwM2J+re6xcYfcElWqzgQiHUBK8bhc3XrDIqXgZWxr7yYuX4HZpnLu40bnv8lVtPHzbW3j002/l0jHC0VyAcJzYlpkAK7xuNE3j4hNb0DTNaoJofvm6+sNs0Ye44ISmcce45tQFXH3K/HH32+e89KSWnPsdx2HlJq4+NfPa9gkcRzBklvzaV8ZXrprHus9eyrWnLrDO0crT1oTUUOV1Fjwuagw4e0Zks8RaZ7K3L+TkINZuOjCud9PGfQNOVU5nMETPSIyRaHKccIC5HuXOJ3fxT7/Zxvo9QdobKp31LJ/7381896ndXPWfzxBLplncFOAP2w4zGkty0TKzv1hnXwjDMPjKw1v58fpO9vSFnBb59uJQe1I8VdURjqdyGj2C2cbc73Fx+ao29gXDvP+uF/jUT1/Jec6g9Zq22gpnHYd95Q5MmjDOOI6sUJXlOAaz9kAJhuIcHorSVlvhtN4JxZNEEyletARgf3+YZCrNnU/uctyTXXAydtGkYRjO9rP9objzvtduOshQJMG+vhALGwM0BHzOa22BbAh4s3q+JR2BBjNE1zcaY+2mA1Ra2wxEp+jVVipEOISS8cFzFzoLCsde2Z+1qJFX//4Kzs4SjslozgpV2XXsY1nSXOXkaMYKVUt1pgni95/ejcfl4oPnLhx3jE9fvpy/ePvycfdnhCNX0OxEfUuNOb63LG92xGRBPsdhOYa+0ZhTJdbRGKCjMeCETK48uQ070mMmx81j2+tNxmI7jr29Ifb2hWir9TMaS/KYFSa0eWX/AGcuamBxUxWdwbAzgWcLx9KWKjTNXMC5qWvIcR8dDQE6GgJ4XGZDy39/32k0VHl512kLuPa0BRyw3MEl1u9nXzDMUzt72Wy5i+wqOjvRbQuHvbnY2HDVIesq/52r53PekkYuX9nG7t6QI7hgTqhVPrMPmu047PwNmAv3JiIxQY4jlc7sp2EeL0wonmJebQUB23HEUry0t59Y0twIal8wxMudA3zj8R3c/exeINM81HYcW/Qh3ve99azfHXTal/SH4k6eI5JI8dCrOnv7QixurqK20uPkOOx+XPUBL36PC69bYySazAnn9Y7EeHzrIeKpNDddtBg4cq6jrIRDynHnlsYqH9efrnBpOG3Zs7GvnAqh2u/B73FR7fdMuGmS3+N2VoGPFSq77XrPcJQHNnZzw1ntOeW3U/EnZyq++M4VzlW3zVjH4fe4uebU+aj6ypyQlk1bVlNC+8p4rDO5eHkLFdZ6mvpKryNai/JUVIEZBpxfV8Hu3lE6gyGuP12xsDGQ097+0FCUvX0hzlrUwMKmAJ3BjAPIFo4Kr7kfy69f1Ymn0k4VV3tDJT6Pi//8wOk8cMv5vPvMdp773GV8872n5SyWPH9pIz6Pi719o3zrD2+i6s0Fa0/u6HHExRYsOxx0aruZZxrbVfmQdZW/sCnAz2+5gC9dvRIgpwuxXXlW7c/svdI9GHYuWPoLcRyu7KoqSKXTjqi11fodwZtXV+FckITiSdbt7MXncXHx8mb2B8NstZ73pDW++oCXar/HcQ0/39DFS/v6ueXelwGzS0N/KO4k8VV9Jbc/tp0dh0dY0lxFXaXXGUd2jkPTNGtvm4SzJqjVavL55uFRqnxuJ9TbNVB4afRMKCvhkHLcueeLV6/k7pvOyTuJTgdN08zqnwnCVDZ2ordizELGlho/8WSa//jDTpKpNLe+bem0zr9iXi03X3zCuPuzcxw2f3/Nyfzqzy/MexzbcRwejtLVH6GxypfTQBJMIbArtxqsclwgbymuzeKmKp7dFSSRMljaUsUVq9p4cW+/M5ne99J+NM3sM7a4KUDXQISHNx1gxbyacTsSntBS7Uzy//H+0/F5XM7eI9ecuoBl1t4iHrcLj9vllDu7XRodjQEWNgb46Yv7ea1rkE9fvpzTFzbwu61mP7KFjQF29Y5a5a7mZHh6Rz2rVS3f+sPOnMT7weFITon0kuYqFjUFeHJHJuk7EI7TUOUl4PcwGss4DtvFBEMxRmPJHAcB8OvXdGdSdY8NVRmZzbOWNFc5v4t5tRW4XRqVXrPr8vO7g5yzuIEV82vpGgg7FX12WKmu0kuttV1yOm3wu22HaK72MRJLMq+2ghXzagiG4k447fZ3n8LlK9v45FuXctOFi6mt8GaFqizHYX3+7W2YbcdxiqqjZyTGmz0jLGutdiq6xHEIxyR1lV4nfDFTmmumFg67Y+1Yx2HnSO7f0MU1py6YMOwzXbKrqmwqfW5aJ9gedqzj6MiTBwH44LkdLG2uYl5dBfWVpigtnGTMS1qqnGqlJc3VXHJSC/FkmvW7g8SSKX76YieXntRqTb5VpNIGm7uHcnqQ2dh5jmWt1ZyxsIH1n7+Md58x/nk2dZVes5tyQyVet4vFTQHC8RTXnb6A957Vzhkd9U5fqGtOnc9gOEEwZDasrPabCzf/9YbTGAwnnFXiaavNx7y63N/PJSe28PzuPnb1jNAzEmUgnKAh4KPK5yZsJaUPDEY4zXIxfaNxPv/LzXz0Ry85x+jqD/Pp+1/jR8+ZISWvK7flSDptJsd9bldOuHG+NZYqv5vRWJK9faOsnFfLosYAiZTBkzt6yE5B2V2cR6IJNnUPcng4xpeuXskHzungxgsW0VjlYyAUJxiKUVfp5eITW7jjQ2fyxXeupKMxQG1lRjgGwwlqKjJu2xSOBIeGo1T53CxurqJ3JMbOw6Msb6uhtcaPz+2ie4IV+aVG9hwXjlref3aHs3BtIs5f2sSPntvnVEbZ2C1JDAP+zyXjnUOx2PmHlurCenk1VftxaWZH3a7+MCdbV8ZjuWxFG5etMPtRXXhCEzeev4hzFjfkfS5k+mmBeZVcW+mh0uvmyR09DEUS9I3GuenCxUAmJwJw7WnjCwHs0NU5Vv6pqYD39tl3rHBWl1+2oo1IIsXt7z4FTdOcUFZrjZ/zljbxX0/tZlfPKEPhTN+xVQtq+bO3LuV763bz2XechM/jIp5K5zgOgEtWtPLj9Z1c/u9P01zts1xOEz63i3A8xeHhKMm0wfK2aqp8boKj5qLNA4MRaydLD3+0dra0y15zkuMuuxw3Tl3A66yrgUzLmIDPw/6guVp7YVPAuQgZDCdYs3oej20xW66YwmHmKX637TAel8ZlJ7XxJ2e0A/A3D2xi5+FRayfN8WuT6iq9jlMaCJu9zGxq/GZ5uF0mbG/oFUmkWN5ajctldrw+Uo5DhEM4avnQeeOT2WO5aFkzr//DlWhabvWR7TguX9nKyvmlC11eeEITX3/3KZy7pLAkv7l63M+hoSj6YIR3rJ435Wsaqnz88/WrJ32OXVlV4/fw/9s78yC36+uAfySt9tDe8rn71utdY2O8Mb5rAwYKBQMb7BgzBszpAAnO1AyTaSYNd1PaDJk0x6RTGuIUJma4TFqTOg1HUjpxwrSAC4RgrmBssP3wgeP79trbP37H/rSS1qu1pJ9qv8+MZqWvpJ/efn8/fZ/e+75jcI3jB585ehAvrN7CL/+widOH1XCe21HRc3lNb0v6kT9BvOi0Gf38nwC/6CI45yl4ria5v/47mut8pbRma083R49rp4/g4ZUfseKtTznXjc7qvQ91/pgh3De7g66jx3jweaekSKPrytt3uMvfOG9prGJQTQXrt+/3F8/VuosZowbx0vvOHoS3t5BWq8p1ozVUxf1M/sZE3LdiE+UxPyx4RDKRsvc0e0Izr67b7ltTdVVxtuw+yAurN3PWqEF+oiU4zc/+tM+JhOqdtwROEMieQ12+BdQYeK8TGbefbhwXWrAI6Rj3/LU0Vtkeh2H0l95KA5xooauntXCn23gqX8RjURZMb80p476lsYoXVm/myNFu3xd9onib9l5nQHAinLbtPURVPMaSG6f540NqK+gcPzyr5TWltZFHFk5j9oR0a2Qg1Cfi3HT2SOZPbaGpzkmiy6Q4Rg6qZnJrAz9/U33f/fBeLr9YNMKt57az6M9P48opjvusoSpOdUUZ+w8d9ZVES2OCZHV5SmLeWxt3svdQF6+u3Z5yzN7Vcb2Fur6qx+IIusyqK8p8pdOaTDC8rtKvd3am1DNtZCMNVXF3E7uMdzftZt22fb68HsnqCg4eOcbGHQdSogY96qridHfDnkNd7Nx/mPqAxTG8vpJ1bt7OsN6Kw92DamlM2B6HYZwI8ViU78yf6G/shsm35p3JODfxMV/WT2syQTTSY3kAzJnYzC0z21m26OyUaLBIJMKPbpialisTfP6iccOyRq8NhAfmjmf2hGai0Qijh9bw3qbd7MzQlGveZOH9zXv4wX/+kUgkc0izx9cvHUuyupzTh9f6iYdrt+1131fJ4JpyPzmuKh7jrQ27ePnDzzh89JhvfUGGcFy/LlScRk9x1PUszF72eCTiREJFoxFGJhPUVpYxIlnFX192Bt+9aiLgWAbd3U4dtDkTm1Pk95SS7jyQ0kLAwwv13n3giLuX0zNXt184mmSinD0Hu1IUR2U86ieejkhWsX3f4ePW1coHpjgMo8CMa6pj2W1n8drdFzF1ZPZ9i1woL4vytUvGpuSm1FfFuX9Oh98zvVSY0Z7kzfU7+WzPobRgh8vPbCIei7B++34enHdmn31gmuqrWHXPxcye0OxH7X2weS/DaiupKIv5i3FlPMpfnDGU32/YydL//oSGRDwlKCC4xxENlBypr+opcx90mXmKY3hdpe++Om/MEC7pGE4k4ihGTyl7UXe3zGxPKbEP+EoJyLjH0ZPodyRtj2NoXSVLbppKdXmMcU21/h7baHd/A3oSEIthddgeh2EUgUgkkjXyaqAsvnB0Xo9XKGaOHsxPfreOwweOpSmOQTUVrLj9XIbUVmT0+/fGcxF6eUIfbNnNiGSVeyxnoT19WC2TWxv45dub0J0H+PsrxtMeqPkVtDiqK2Js3nXQcQ1VxXsUR13AVeUqqaBCvn9OR0b5OprraBuUYMH0EWnPJVMUR/r/6s3N9n2H2XOwK806m9DSwBv3z6KiLMaxY92URSO+mwqcvay2QYm0AoyFoOQVh4hcAPwd8A7wtKr+JlSBDMPIientSeKxCEeOdqdsFnsMxH3nWRwbth9g2sjUaLAxQ2uZ6PaB+bO2Rq6b3ppS8DKYANg5vol7f74acBL4mhsqmdGeZGagsGXCVVKt/bDkZk9oZvaE5ozPBSO2Bldn2uPo+Z+AFIvDw+vYGY1GuOfycUwONCCb0NLAb75+4XFlzAcFdVWJyKMislVEVvcav0xEPhCRNSJy53EO0w3sBSqB9EpuhmGUNInyMj9p8Hh5Of0lWJnAy43xNpzHDq9h0ogGFp0/iu9dNYloNMKQGifPAUgJbLhisvj1qJzyHjGWLTo7pTSOZ3H0R3H0RbKmb4vDc1V94vY36W1x9Obmme1+o7RiU2iL46fAPwGPeQMiEgMeAmbhKIJVIrICiAEP9nr/LcDvVHWliAwDvg9cX2CZDcPIM+eOHsyr67bnT3EEKhN4vn0vIqujqZ54LMpdnx/nv8bLc1i3bV/K3kNNRRnzpgiPv7I+q2yJPCmO2ooy3/LKmMfhKor1bg2vbH1oSoGCKg5V/a2ItPUang6sUdW1ACLyNDBXVR8EZvdxuB1AVieoiNwG3ObePxGxDcPIMxd3DOMf/+tDRg1Ob4U7EIJlW7zaX9Pbkzz5pRkZqyB7r1u3bV9aKPUXz2nj+bc3+1UI0j/LsUhONOggEnEKRm7dk97xEqCmvIxIpKeJVX+TTMMgjD0OATYEHm8EZmR9sciVwKVAA471khFVXQIsAejs7MzezcUwjKIzrqmO1X97qe+jP1ESvfq9gLMwnzN6cLa3+JZJPJaqOEYPreX1+2ZlfV9rMkF1eYzThpx42ZpkdTnb9x329zOCRKNO58nP9hzi8glNjGsKP5Q8G2EojkyZU1kXelVdDizvz4FFZA4wp62tbWCSGYZRMPKlNKDH4ohG0rPNs9E+OEEsGslZjlkdw1h178UnXLgTnMivHfvLMyatgpP/0ZpM8N35E7O+phQIQ3FsBIKxai3Ap/k4sKr+AvhFZ2fnl/NxPMMwShPP4miqr0rLl8jGDWeNZOrIxpTulP0hEonkRWkAzGgflFadOMjjX5pBTUVZWtHOUiMMxbEKGCMi7YACC4Dr8nFgszgM49TAW8gzdV3s6z1ea+CwuCND07Ag/cllKQUKHY77FPA/wFgR2Sgit6pqF3A78CLwHvCMqr6Tj8+zfhyGcWrg9ckotSz5U4VCR1Vdm2X8OeC5fH+eWRyGcepwZ+cZfqKfUVxOqlpVZnEYxqnDwnPaQkuAO9U5qRSHYRiGUXhKvlZVLpiryjAMo/CcVBaHuaoMwzAKz0mlOAzDMIzCY64qwzAMIydOKovDXFWGYRiF56RSHIZhGEbhMcVhGIZh5MRJuccB7BaRDwd4mMHAtvxJlVdMtoFhsg0Mk21g/H+WbWS/jtLd3W23wK25ufl/w5bBZDPZSuFmspls2W7mqjIMwzBywhSHYRiGkROmONJZErYAfWCyDQyTbWCYbAPjpJct0t1t7bkNwzCM/mMWh2EYhpETpjgMwzCMnDip8jhOFBG5DPghEAP+RVW/HaIsI4DHgOHAMWCJqv5QRL4JfBn4zH3p3W5HxWLL9zGwBzgKdKnqNBFJAsuANuBj4GpV3VFkuca6MniMAu4HGghh3kTkUWA2sFVVx7tjGedJRCI419/ngf3AF1X1jSLL9g84uVCHgY+Am1V1p4i04bR6/sB9+yuq+pUiy/ZNspxDEbkLuBXnerxDVV8ssmzLgLHuSxqAnao6KYR5y7Zu5PWaM4vDRURiwENAJ9ABXCsiHSGK1AV8TVXHAWcBiwPy/EBVJ7m3oiuNABe6MkxzH98JvKSqY4CX3MdFRVU/8OYGmIrzZXjWfTqMefspcFmvsWzz1AmMcW+3AT8KQbZfA+NVdQLwR+CuwHMfBeavYItfH7JBhnPofi8WAJ9z3/PP7ve5aLKp6jWB6+7fgOWBp4s5b9nWjbxec6Y4epgOrFHVtap6GHgamBuWMKq6ydP8qroH51eLhCVPP5kLLHXvLwWuCFEWgItwvrSfhCWAqv4W2N5rONs8zQUeU9VuVX0FaBCRpmLKpqq/UtUu9+ErQEuhPr8vssxbNuYCT6vqIVVdB6zB+T4XXTb3F/zVwFOF+vy+6GPdyOs1Z4qjBwE2BB5vpEQWatfcnQy86g7dLiJ/EJFHRaQxJLG6gV+JyOsicps7NkxVN4FzAQNDQ5LNYwGpX+BSmDfIPk+ldg3eAjwfeNwuIm+KyEoROS8kmTKdw1Kat/OALaoaLHkUyrz1Wjfyes2Z4ughkmEs9FhlEanBMX2/qqq7cUzJ04BJwCbgeyGJNlNVp+CYuotF5PyQ5MiIiJQDXwB+5g6Vyrz1RclcgyJyD47b4wl3aBPQqqqTgb8CnhSRYvcvyHYOS2begGtJ/bESyrxlWDeyMaC5M8XRw0ZgROBxC/BpSLIAICJxnJP/hKouB1DVLap6VFWPAT+hgCZ5X6jqp+7frTh7CNOBLZ6Z6/7dGoZsLp3AG6q6BUpn3lyyzVNJXIMishBn8/d6Ve0GcN1Af3Lvv46zcX56MeXq4xyWyryVAVcSCM4IY94yrRvk+ZozxdHDKmCMiLS7v1YXACvCEsb1lT4CvKeq3w+MB/2P84DVIchWLSK13n3gEleOFcBC92ULgX8vtmwBUn75lcK8Bcg2TyuAm0QkIiJnAbs890KxcCMLvwF8QVX3B8aHeBvOIjIKZzN1bZFly3YOVwALRKRCRNpd2V4rpmwuFwPvq+pGb6DY85Zt3SDP15yF47qoapeI3A68iBOO+6iqvhOiSDOBG4G3ReT37tjdONFek3DMyY+BRSHINgx4VkTAuYaeVNUXRGQV8IyI3AqsB64KQTZEJAHMInVuvhPGvInIU8AFwGAR2Qj8DfBtMs/TczhhkWtwosFuDkG2u4AK4Nfu+fXCR88HHhCRLpyQ16+oan83r/Ml2wWZzqGqviMizwDv4rjXFqvq0WLKpqqPkL6nBkWeN7KvG3m95qzkiGEYhpET5qoyDMMwcsIUh2EYhpETpjgMwzCMnDDFYRiGYeSEKQ7DMAwjJ0xxGEaJISIXiMh/hC2HYWTDFIdhGIaRE5YAaBgDRERuAO4AynEKyf0lsAv4MXAhsANYoKqfuYlrDwMJnLITt7j9EEa740NwEsS8xKwaEflXYDzwOnCDV/7DMMLGLA7DGAAiMg64BqfY4yScRf96oBqnRtYUYCVOxjM4zXW+4fa5eDsw/gTwkKpOBM7BKYoHTlXTr+L0hhmFkxFsGCWBWRyGMTAuwmkUtcotzVGFUzjuGD1F7h4HlotIPdCgqivd8aXAz9x6X6KqzwKo6kEA93iveTWP3NIRbcDLhf+3DOP4mOIwjIERAZaqarBDHiJyX6/X9eVeylTS2uNQ4P5R7LtqlBDmqjKMgfESMF9EhoLTR1xERuJ8p+a7r7kOeFlVdwE7Ak18bgRWun0SNorIFe4xKtwCjYZR0tivGMMYAKr6rojci9MFMQocARYD+4DPicjrOBvl17hvWQg87CqGtfRUIb0R+LGIPOAeI5SKwoaRC1Yd1zDyiIjsVdWasOUwjEJirirDMAwjJ8ziMAzDMHLCLA7DMAwjJ0xxGIZhGDlhisMwDMPICVMchmEYRk6Y4jAMwzBy4v8AWjJzmRaIJgUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1c4c7e9e10>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "## summarize history for loss\n",
    "## Plotting the loss with the number of iterations\n",
    "plt.semilogy(history.history['loss'])\n",
    "plt.semilogy(history.history['val_loss'])\n",
    "plt.title('model loss')\n",
    "plt.ylabel('loss')\n",
    "plt.xlabel('epoch')\n",
    "plt.legend(['train', 'test'], loc='upper left')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 375,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEWCAYAAACXGLsWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4xLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvAOZPmwAAIABJREFUeJzt3XmYVOWZ9/Fv9QLd7NAgJDcoKCSGOMYF0WQmgsYkErcoatS4kkSNcdRkfCfRyagvidE3mky8HEbjQhSNK9GMRgzBDc04GreoMQZFwdA3iNDse1fVef94TkF1dVVXAV1d3c3vc11cnLXq7tPV565nOc+TiKIIERGRtlRVOgAREen8lCxERKQoJQsRESlKyUJERIpSshARkaKULEREpCglCxHAzO4wsx+XeOxCMzui3DGJdCZKFiIiUpSShUg3YmY1lY5Buid9sKTLMLOFwDTgDGAv4D7gcuAO4J+AF4GT3H1lfPyxwDWAAX8Gvu3ub8f79gduB8YAs4AWQxmY2dHAj4GRwF+B8939jRJiPCo+by9gNXC7u1+Vtf+fgJ8CY4G1wL+7+x1mVh+fdyIwAHgT+CJwMHC3uw/PuQ7fdPcnzOwqYB9gE3As8D0zewO4AfgUsBH4DfA9d98Sn/9p4BfAgUBzfOx04H1ghLs3xccdCPwe+Li7Nxf72aV7U8lCuprJhJvoJ4BjgMcJCWMw4fN8EYCZfQK4F7gEGEJICI+aWQ8z6wH8FrgLGAQ8GL8u8bkHEG6e5wENwC+BR8ysZwnxrQfOJNzwjwK+bWZfjV939zjeG+OY9iMkMYDrCTfvz8Ux/SuQLvGaHAfMjN/z10AK+G58TT4LfAG4II6hL/AEcRIARgNPuvuHwDPAyVmvezpwnxKFgEoW0vXc6O5LAczsOeAjd38tXn+YcGME+BrwmLvPifddD1xMuBmngVrgF+4eATPN7HtZ7/Et4Jfu/mK8fqeZXQ4cAsxtKzh3fyZr9Q0zuxeYQEhOXweecPd74/1NQJOZVQFTgEPc3eN9z8dxl3JN/tfdfxsvbwReydq30Mx+GcfwC+Bo4EN3/1m8fxOhRAZwJyHZ3mRm1cCphNKKiJKFdDlLs5Y35lnvEy9/HPggs8Pd02a2iFAllQqbPLvq6YOs5T2As8zsn7O29Yhfs01mdjBwLaFqqAfQk1ByARgBvJfntMFAXYF9pViUE8MngJ8D44BehL/zTAIpFAPAfwM3m9mehJLbanf/0w7GJN2MqqGku1pMuOkDYGYJwo3SgSVhkyWyjt89a3kRcLW7D8j61yurRNCWe4BHCHX//YGbgcz7LCK0ZeRaTviGn2/fesINP/NzVBOqsLLlDh19E/A3YIy79yNU0xWLAXffBDxAKAGdQaimEwFUspDu6wHgB2b2BeBZQhXUZuLqHSAJXGRm0whVLeOBp+N9twIPm9kTwJ8IN+uJwLPuvrbI+/YFVrj7JjMbD5wG/CHe92vgcjM7GXgI6E9IKn82s+nAz83sDEJpaTzwKvAOUBc3nP+BcOMv1nbSF1gDrDOzvYFvA8vifb+L3+cSQlLpAYzNqnKbEf/bDfi3Iu8juxCVLKRbcvd5hAbaGwnf3I8BjnH3LXGvoBOAs4GVhPaNh7LOfZnQbvGf8f758bGluACYamZrgSsISSvzun8HvgL8C7CC0Lj9mXj3pYQeUC/F+/4fUOXuq+PXvI1QKloPNBaJ4VJCklpLSHz3Z8WwltBB4BjgQ+Bd4LCs/f9DaNN51d0Xlvgzyy4gocmPRCSbmT0F3OPut1U6Fuk8VA0lIluZ2UHAAYTuuCJbqRpKRAAwszsJz2BcUkLbjOxiVA0lIiJFqWQhIiJFdZs2i3322ScaMWJEpcMQEelS3njjjeXunvvsTivdJlmMGDGCxx9/vNJhiIh0KWb2QfGjVA0lIiIlULIQEZGilCxERKSobtNmkU9zczONjY1s2rSp0qGUXV1dHcOHD6e2trbSoYhIN9Stk0VjYyN9+/Zl5MiRJBKJ4id0UVEU0dTURGNjI6NGjap0OCLSDZUtWcSjaB5NmJxmnzz7E4TpHL8CbADOdvdX431nAT+MD/2xu9+5IzFs2rSp2ycKgEQiQUNDA8uWLSt+sIjIDihnm8UdwJFt7J9EmP94DHAuYbhkzGwQcCVh7uHxwJVmNnBHg+juiSJjV/k5RaQyylaycPdnzWxkG4ccB8yIZyt7wcwGmNnHCPMGzHH3FQBmNoeQdEqZeKZdJFNpmtZvoRIjofStq6F3z/BriaKIlRua2ZIsbSrmNRub+fkf5pUzPJGuJ5Hg+P2NUYN7A9CcSjP9jwtYvzlZ4cDaz7D+9Zx28O7FD9wJlWyzMFpOB9kYbyu0vfULmJ1LKJWUOldxSVZtbGbpmvZpFF+3dg3PzH6Mo088taTjm9Yl+MSwvkw5+yyuvu4/WJUs/Ve0dlOSG59eVPxAkV1IFMHvXl/M45d8np411by0YAXXPP43ALpLgXy/EQO6dbLI92uK2tjeirvfAtwCMGnSpHYrByRTEQlgH+u/09U7ixat5YlHHuTyS77dYnsqlaK6urrFts3NKd75aB0frt7EHXfO4J2la6mvTTB6tz4lxfH22noWXHPUTsUr0t08M+8jzv7VS9z+xwVcMHE0jSs3AvDcvx7GiEG9ipwtGZVMFo2EOZEzhhPmTW4kVEVlb3+mw6ICkuk01VVV7dIO8JOf/IQPPviAL37xi9TW1tKrVy+GDh3KW2+9xTPPPMOUKVNYvHgxmzdv5hvf+AZHHHMiH63dxHFfmsANdzzA0HqYePLRjB8/npdffplhw4Yxffp06uvr2+EnFen+Jn5yN7786aHc+OR8Tj1odxpXbSSRgGH96yodWpdSyWTxCHChmd1HaMxe7e5LzGw28JOsRu0vAZft7Jv930ff4q+L15R07KbmFBFQX1vd5nFjP96PK4/5dJvHXH755cybN485c+bw/PPPc+aZZ/LUU0+x++6hyPizn/2MgQMHsnHjRo466igmTfoKu/XrRVUigQ2opzZqZsGCBUybNo3rrruO8847j1mzZjF58uSSfhYRgTM/O5LZby3l7SVr8JUbGdq3jtpqPZO8PcrZdfZeQglhsJk1Eno41QK4+83ALEK32fmErrPnxPtWmNmPCHMRA0zNNHZ3lEJ1Ye1hv/3225ooAKZPn751AMTFixezcOECDjzwQKqrEgzo1YP165sZMWIE++wTeh/vu+++LFqkdgmR7bHnkNC4/f7y9fiqDdhAlcy3Vzl7Q7XZohv3gvpOgX3TgentGU+xEkC2eR+uob62ht0b2r8+s1evba/5/PPP89xzz/Hoo49SX1/PiSeeyObNm1ud07Nnz63L1dXVu8QT6SLtaWjfOuprq3l/2Xp81Ub2H7HDvfF3WSqH5ZFMRdRUt0/Zonfv3qxbty7vvrVr19K/f3/q6+uZP38+r776aru8p4i0VFWVYNTg3sxfto4lqzapZLEDuvVwHzsinY5IRRE1Ve2TLAYNGsRBBx3E4YcfTl1dHYMHD966b+LEidx1110cccQR7LnnnhxwwAHt8p4i0tqoIb2ZO28ZyXTExwcoWWwvJYscyXTogVvdTiULgGnTpuXd3rNnT+6+++68+1588UUgJJunnnpq6/bzzz+/3eIS2ZXsNbg3j72xBIDhShbbTckiRzIdnpauqVINXYfasAKSrdtrRHZaIgF9hjJqSG9qSDKItezeYzWsSXV8LPUDoLZrJiolixyZkkV7VUNJCT54Hn41qdJRSHc24fuMGn0BN9f+B0dUvwYzKhTHoL3goq7ZNqlkkSOVipNFO1ZDSRH+Svh/0k+hukdlY5Hu55lrYPm7jDqkN/0Ti3mT0fzD0Rd2fBwLn4O//AY2roT6rtcbS8kih6qhKqBpPvRqgIPPq3Qk0h39+dewoYn+9bUkqtbxl57j+Idx53R8HH2HhWTR9B4MH9fx77+TdEfMkUxHVCUSqBaqAy2fDw2jKx2FdFe9GmDjCkgl6cd6BjQMq0wcmc/48ncr8/47SckiRzIVus1qfogO1KRkIWVUPwg2rIRNqwD4/Gc+WZk4BuwBierwee+ClCxyJNNRu3abXb16NXfccccOnXvrrbeycePGdoulU9q8FtZ9qGQh5dNrUChZbIhHDaofVJk4anrAwJFKFt1FMpVu1/aKNWvWMGPGjnW9uO2227p/ssj84ShZSLnUD4TmDbDGw3qvCjYuN4zusslCDdw5kumIutr2K1lkD1F+6KGHMnjwYB599FG2bNnCkUceyaWXXsqGDRs477zzWLJkCel0mosvvpjly5ezdOlSTjrpJAYOHMjMmTPbLaZOpem98P/gMZWNQ7qvXg3h/8xNOrNeCQ2jYcGzkE5DF+tEs+ski8d/AB++WfSw3bckwzMWNW0PTw7AsH+ASde2eUj2EOVz587lscce47HHHiOKIs4++2xeeOEFmpqaGDZsGHfddRcQSiP9+vXjlltu4cEHH2TQoAoVmzvC8neBBAwcVelIpLvqFf/9ZL6YVKoaCmDwaEhuDKWcASOKH9+JdK3UVmYREVFE2Rq3586dy9y5c/nSl77El7/8Zd577z0WLFjA3nvvzXPPPcfVV1/Niy++SL9+/cry/p1S0/zwR1OriWikTDLJYWvJooLJIlPd2gWronadkkWREgBAlI5Yv2QhtTVJqCkxj2a+rRSycgkkt0DTe0QbV3LhOV/jjJOOa3XY4/fcxFPPvcA1P7qSCZ89iO+efzakk7BiAUQrS4tl/TK456rSju0sFr0IH9cAilJGvbKSRVUt9OhTuVgaxmyLZa/DWu579S742+928HVHw5ev3rnYith1kkUJ0lHEEFZDuhrSte3ymr3ra1m3YT2km5l4yIFc91+/4oRJh9G7Vz1Lli6jtraGZDLFgP79mPyVw+hdV8sDj8yGdDN9etWzbt0aBvXvXeIPkIK1S9ol7g4zYHf4zCmVjkK6s0zJYtUH0HtIGCuqUvoOg9re+UsW/3MDbFge/ia2Vwc8Ea5kkSUdQTURm3oMoL6hfeoTBw2Bgw7+HId/7QIOO+wwvnrSqRw75VIgTIR04403snDJQn588fdIJBLU1tZyzTXXwJC9+fpZUzj9oivZbbfdSmvgXh7Bec+2S9wi3UamZBGlK9u4DSFRNezVOlmkmmHlAvjcRXDElZWJrQgliyzpKApfOtr5m0fuEOXf/OY3W6yPHDmSiRMntjpvypQpTJkypV1jEdnl1PQMVU9b1lW2cTtj8BhofLnltlV/D9XOnbhXoBq4s0RRGBcqUbYZuEWkIjJJopLPWGQ0jA7JIXtI/i7wvJGSRZZ0PDy5hvoQ6WYySaIzlCwaRgMRrHh/27bMeFFKFpUTRdH2H9sFk8X2/Jwiu5xMW0Wl2ywgf/fZpvkhkVWyW28R3TpZ1NXV0dTUVPKNNHNcItG1LksURTQ1NVFXp2cVRPLaWg3VCW7GhZJFJy5VQDdv4B4+fDiNjY0sW7aspOM3bt5C/caPSNU1U123oszRta+6ujqGDx9e6TBEOqdMkugM1VB1/aDP0DA0f0bTfNjr8MrFVIJunSxqa2sZNar0YSQefe5ljnnyZFYc/lMG7a+JeES6jc5UsoCWAwpuXheej2rYq7IxFdGtk8X2am7eAkBtjab2FOlWOlObBYTE8Paj8M7s0DMKVA3VlWzeEieLHkoWIt3KoD2hqgb6d5Kq2mH7wqsz4J6Tt20buk/l4imBkkWW5uZmIFRfiUg3MvoL8L23oc9ulY4kGDcFRowPD+IB1A1QNVRXkqmGqq5RshDpVhKJzpMoAKqq4WOfqXQU26Vr9REts0yyoErJQkQkm5JFlmRcDUWVClwiItmULLIkt5YslCxERLIpWWTZVrIoYUpVEZFdSFm/QpvZkcANQDVwm7tfm7N/D2A6MARYAZzu7o3xvp8CRxES2hzgYncv6wBIzck4WVSrzUJEJFvZShZmVg1MAyYBY4FTzWxszmHXAzPcfV9gKnBNfO7ngH8E9gX2AQ4CJpQr1oxUUm0WIiL5lLMaajww393fd/ctwH1A7uTTY4En4+Wns/ZHQB3QA+gJ1AJLyxgrAOmk2ixERPIpZ7IwYFHWemO8LdvrwOR4+Xigr5k1uPv/EpLHkvjfbHd/u9UbmJ1rZi+b2ctNTU07HXAyqTYLEZF8ypks8k0KkdvmcCkwwcxeI1QzOZA0s9HAp4DhhARzuJkdmvti7n6Lu49z93ENDTs/5ks6GT9NqecsRERaKGd9SyMwImt9OLA4+wB3XwycAGBmfYDJ7r7azM4FXnD3dfG+x4FDgGfLGC+plNosRETyKWfJ4iVgjJmNMrMewCnAI9kHmNlgM8vEcBmhZxTA3wkljhozqyWUOlpVQ7W3SMlCRCSvsiULd08CFwKzCTf6B9z9LTObambHxodNBOaZ2TvAUODqePtM4D3gTUK7xuvu/mi5Ys1Iq81CRCSvsn6FdvdZwKycbVdkLc8kJIbc81JAh84+lEylIUqFFT1nISLSgp7gjm1KpqkhHVZUDSUi0oKSRWzjlhQ1ZHpDKVmIiGRTsohtak5llSzUZiEikk3JIrapOUU1cZuFnrMQEWlBySK2sUXJQtVQIiLZlCxim5rTarMQESlAySK2sTlFTUJtFiIi+ShZxDJtFlGiJkzuLiIiWylZxDK9oSJVQYmItKJkEdv6nIWqoEREWlGyiIVqqLSG+hARyUPJIraxOU0NKRKqhhIRaUXJIhbaLFLqNisikoeSRWxLKk2PqrRKFiIieShZxFLpiFrSUK1kISKSS8kilkxF1CRUDSUiko+SRSyVTlOrZCEikpeSRSwVxdVQShYiIq0oWcRS6SiMDaVkISLSipJFLJmK1HVWRKQAJYtYKFkoWYiI5KNkEUtFUZj8SMlCRKQVJYtYMh1XQ+k5CxGRVpQsYqmUGrhFRApRsohtLVkoWYiItKJkEUtHEdVKFiIieSlZxFSyEBEpTMkilkqnw+RHShYiIq0oWcTCQ3lJJQsRkTyULGJb2yzUdVZEpBUli1gyHVEdqRpKRCQfJYtYKh1RpQZuEZG8SrozmtlvgOnA4+6eLvXFzexI4AagGrjN3a/N2b9H/LpDgBXA6e7eGO/bHbgNGAFEwFfcfWGp77291GYhIlJYqSWLm4DTgHfN7Foz27vYCWZWDUwDJgFjgVPNbGzOYdcDM9x9X2AqcE3WvhnAde7+KWA88FGJse6QdBRRFalkISKST0l3Rnd/AnjCzPoDpwJzzGwRcCtwt7s35zltPDDf3d8HMLP7gOOAv2YdMxb4brz8NPDb+NixQI27z4nff932/mDbK5nWQ3kiIoWU3GZhZg3A2cA3gdcI1UsHAHMKnQIsylpvjLdlex2YHC8fD/SN3+cTwCoze8jMXjOz6+KSStmk0hHVKlmIiORVapvFQ8DewF3AMe6+JN51v5m9XOC0RJ5tUc76pcB/mtnZwLOAA8k4rs8D+wN/B+4nJKrbc+I6Fzg3Xi7lRykolUpSpYfyRETyKvXO+J/u/lS+He4+rsA5jYTG6YzhwOKccxcDJwCYWR9gsruvNrNG4LWsKqzfAoeQkyzc/RbgFoBJkyblJqLtk47b7fWchYhIK6VWQ33KzAZkVsxsoJldUOScl4AxZjbKzHoApwCPZB9gZoPNLBPDZYSeUZlzB5rZkHj9cFq2dbS/dNzsopKFiEgrpSaLb7n7qsyKu68EvtXWCe6eBC4EZgNvAw+4+1tmNtXMjo0PmwjMM7N3gKHA1fG5KUIV1ZNm9iahSuvWkn+qHZBIJ8OCkoWISCul3hmrzCzh7hFs7Rbbo9hJ7j4LmJWz7Yqs5ZnAzALnzgH2LTG+nRYpWYiIFFTqnXE28ICZ3UxopD4f+H3ZoqqARJQKC0oWIiKtlHpn/D5wHvBtQpXQHwhPV3cf6WSolFOyEBFppdSH8tKEp7hvKm84lZNQshARKajU5yzGEIbiGAvUZba7+55liqvDJdKqhhIRKaTU3lC/IpQqksBhhHGb7ipXUB0tiiISUdzAXV1b2WBERDqhUpNFvbs/CSTc/QN3v4rw7EO3kI4IU6oCVJV1VBERkS6p1DqXTfHDc++a2YWEYTl2K19YHSuZTlOLus6KiBRSasniEqAXcBFwIHA6cFa5gupoqXSUVbJQNZSISK6iX6PjB/BOdvf/A6wDzil7VB0slY6oQQ3cIiKFFC1ZxENvHGhm+UaR7RZSmbksQG0WIiJ5lPo1+jXgv83sQWB9ZqO7P1SWqDpYMh1Rm1DJQkSkkFLvjIOAJlr2gIqAbpEsWrRZqOusiEgrpT7B3e3aKbKpzUJEpG2lPsH9K1rPcoe7T2n3iCqgZbJQm4WISK5Sv0b/Lmu5jjBf9uICx3Y5yRYN3CpZiIjkKrUa6jfZ62Z2L/BEWSKqgFQ6TY2esxARKajUh/JyjQF2b89AKimVRiULEZE2lNpmsZaWbRYfEua46BbCcB9qsxARKaTUaqi+5Q6kklLpiGo9ZyEiUlBJ1VBmdryZ9c9aH2BmXy1fWB0rmY62tVnoOQsRkVZKbbO40t1XZ1bcfRVwZXlC6nhp9YYSEWlTqcki33Hd5q6aTEdqsxARaUOpN/yXzeznwDRCQ/c/A6+ULaoOllLJQkSkTaWWLP4Z2ALcDzwAbAS+U66gOlqLNgs9ZyEi0kqpvaHWAz8ocywVozYLEZG2ldobao6ZDchaH2hms8sXVsdqOUS52ixERHKVWg01OO4BBYC7r6QbzcGdSqepJkWUqIZEt53jSURkh5WaLNJmtnV4DzMbSZ5RaLuqZDzqbKQqKBGRvEq9O/4b8EczmxuvHwqcW56QOl4q03VWjdsiInmVVLJw998D44B5hB5R/0LoEdUthPkskkR6eltEJK9SBxL8JnAxMBz4M3AI8L+0nGa1y9radTahxm0RkXxKbbO4GDgI+MDdDwP2B5aVLaoOtnWmPJUsRETyKjVZbHL3TQBm1tPd/wZ8snxhdaxUOqImkdQzFiIiBZR6d2yMn7P4LTDHzFZSwrSqZnYkcANQDdzm7tfm7N8DmA4MAVYAp7t7Y9b+fsDbwMPufmGJsW63rQ3cKlmIiORVagP38e6+yt2vAv4duB1oc4hyM6smjCU1CRgLnGpmY3MOux6Y4e77AlOBa3L2/wiYS5llus6qN5SISH7bXe/i7qXevMcD8939fQAzuw84Dvhr1jFjge/Gy08TSi7Exx8IDAUyPbHKJszBnYJqVUOJiORTzrujAYuy1huBg3OOeR2YTKiqOh7oa2YNwErgZ8AZwBcKvoHZucTPe5jZDgeaSkMNKRIqWYiI5FVqA/eOyDduRu5T35cCE8zsNWAC4EASuACY5e6LaIO73+Lu49x9XENDww4Huq1koWQhIpJPOUsWjcCIrPXh5DSKu/ti4AQAM+sDTHb31Wb2WeDzZnYB0AfoYWbr3L0sI99mJj9KqBpKRCSvct4dXwLGmNkoQonhFOC07APMbDCwwt3TwGWEnlG4+9ezjjkbGFeuRAHbus4mVLIQEcmrbNVQ7p4ELgRmE7q/PuDub5nZVDM7Nj5sIjDPzN4hNGZfXa542hK6zqb1nIWISAFlvTu6+yxgVs62K7KWZwIzi7zGHcAd7R/dNqnMfBZq4BYRyaucDdxdRlIP5YmItEnJgkybRUrVUCIiBShZoOE+RESKUbIgVENVa7gPEZGClCwID+WFmfI0n4WISD5KFmQGEkyqGkpEpAAlC7ImP1I1lIhIXkoWaKY8EZFilCwIyaIazZQnIlKIkgWZ3lAa7kNEpBAlCyCdSoVkoWooEZG8lCwA0s3hf5UsRETyUrIASCXD/ypZiIjkpWQBkN4S/lfXWRGRvJQsANJxyULVUCIieSlZQFY1lJKFiEg+ShZAYmvJQtVQIiL5KFnAtt5QauAWEclLyYLskoWqoURE8lGygG0N3CpZiIjkpWQBVKlkISLSJiULyOo6q5KFiEg+ShZAVZRp4FbJQkQkHyUL1HVWRKQYJQuykoUauEVE8lKyAKoiNXCLiLRFyQJIKFmIiLRJyQKoVjWUiEiblCzILlkoWYiI5KNkQVabhbrOiojkpWQBJKJUWFCbhYhIXrt8skinI2rJJAtVQ4mI5FPWr9JmdiRwA1AN3Obu1+bs3wOYDgwBVgCnu3ujme0H3AT0A1LA1e5+fzliTKYjalADt4hIW8pWsjCzamAaMAkYC5xqZmNzDrsemOHu+wJTgWvi7RuAM93908CRwC/MbEA54ky1KFmoGkpEJJ9yVkONB+a7+/vuvgW4Dzgu55ixwJPx8tOZ/e7+jru/Gy8vBj4ilD7aXTKdpiaTLFSyEBHJq5zJwoBFWeuN8bZsrwOT4+Xjgb5m1tDiRczGAz2A91q9gdm5Zvaymb3c1NS0Q0Gm01CtkoWISJvKmSwSebZFOeuXAhPM7DVgAuCQaUAAM/sYcBdwjrunc1/M3W9x93HuPq6hoSF3d0mS6bQauEVEiijnV+lGYETW+nBgcfYBcRXTCQBm1geY7O6r4/V+wGPAD939hXIFWd+jmgmjBxD9vYpE1S7fOUxEJK9yJouXgDFmNopQYjgFOC37ADMbDKyISw2XEXpGYWY9gIcJjd8PljFGevWo4YDhfaBRpQoRkULK9lXa3ZPAhcBs4G3gAXd/y8ymmtmx8WETgXlm9g4wFLg63n4ycChwtpn9Of63X7liJZVU47aISBvK2qLr7rOAWTnbrshangnMzHPe3cDd5YythXRSjdsiIm1QJT1AulnJQkSkDUoWAKlmVUOJiLRByQLiaiglCxGRQpQsIC5ZqBpKRKQQJQuI2yxUshARKUTJAiCdUgO3iEgblCxA1VAiIkUoWYCqoUREilCyAHWdFREpQskC9AS3iEgRShagZCEiUoSSBagaSkSkCCUL0BPcIiJFKFmAus6KiBShZAHqOisiUoSSBYRqKLVZiIgUpGQBYaa8qupKRyEi0mkpWYCqoUREilCyAHWdFREpQskC1HVWRKQIJQuIG7jVdVZEpBAlCwjVUBruQ0SkICWLKIIopWooEZE2KFmkmsP/qoYSESlIySIdJwuVLEREClKy2FqyULIQESlEySKdCv+rgVtEpCAli6pqGPvJF+xVAAAIGElEQVRVaNir0pGIiHRa+jpdPwBOvrPSUYiIdGoqWYiISFFKFiIiUpSShYiIFFXWNgszOxK4AagGbnP3a3P27wFMB4YAK4DT3b0x3ncW8MP40B+7uxoWREQqpGwlCzOrBqYBk4CxwKlmNjbnsOuBGe6+LzAVuCY+dxBwJXAwMB640swGlitWERFpWzmrocYD8939fXffAtwHHJdzzFjgyXj56az9XwbmuPsKd18JzAGOLGOsIiLShnJWQxmwKGu9kVBSyPY6MJlQVXU80NfMGgqca63ewOxc4Nx4ud0CFxGRlspZskjk2RblrF8KTDCz14AJgAPJEs/F3W9x93HuPq6hoWFn4xURkQLKWbJoBEZkrQ8HFmcf4O6LgRMAzKwPMNndV5tZIzAx59xn2nqzN954Y7mZfbAT8Q4Glu/E+eWk2HaMYtsxim3HdNXY9ijlBcqZLF4CxpjZKEKJ4RTgtOwDzGwwsMLd08BlhJ5RALOBn2Q1an8p3l+Quw/ZmWDN7GV3H7czr1Euim3HKLYdo9h2THePrWzVUO6eBC4k3PjfBh5w97fMbKqZHRsfNhGYZ2bvAEOBq+NzVwA/IiScl4Cp8TYREamAsj5n4e6zgFk5267IWp4JzCxw7nS2lTRERKSC9AT3NrdUOoA2KLYdo9h2jGLbMd06tkQUtepkJCIi0oJKFiIiUpSShYiIFLXLT35UbLDDDo5lBDADGAakgVvc/QYzuwr4FrAsPvTyuPNAR8e3EFgLpICku4+Lx/G6HxgJLAROjodo6ci4PhnHkLEncAUwgApcNzObDhwNfOTu+8Tb8l4nM0sQPn9fATYAZ7v7qx0c23XAMcAW4D3gHHdfZWYjCT0Z58Wnv+Du53dwbFdR4HdoZpcB3yB8Hi9y99kdHNv9wCfjQwYAq9x9vwpct0L3jXb9zO3SJYsSBzvsSEngX9z9U8AhwHey4vkPd98v/tfhiSLLYXEMmT7bPwCedPcxhHG+ftDRAbn7vMy1AQ4k/AE8HO+uxHW7g9ZjmRW6TpOAMfG/c4GbKhDbHGCfeEDPd2j5TNN7WdevbDe8NmKDPL/D+O/iFODT8Tn/Ff89d1hs7v61rM/db4CHsnZ35HUrdN9o18/cLp0sKG2www7j7ksyGd7d1xK+nXT2Qa+OAzLDx98JfLWCsQB8gfCHujNP8+8Ud3+WMOR+tkLX6TjCyMuRu78ADDCzj3VkbO7+h/i5KIAXCCMmdLgC162Q44D73H2zuy8A5hP+njs8tvib+snAveV6/7a0cd9o18/crp4sShqwsBLiouz+wIvxpgvN7A0zm17B4doj4A9m9ko8iCPAUHdfAuFDC+xWodgyTqHlH21nuG5Q+Dp1ts/gFODxrPVRZvaamc01s89XKKZ8v8POdN0+Dyx193eztlXkuuXcN9r1M7erJ4uSBizsaPE4Wb8BLnH3NYRi4l7AfsAS4GcVCu0f3f0AQjH2O2Z2aIXiyMvMegDHAg/GmzrLdWtLp/kMmtm/Eao0fh1vWgLs7u77A98D7jGzfh0cVqHfYae5bsCptPyCUpHrlue+UcgOXbtdPVkUHeywo5lZLeEX/mt3fwjA3Ze6eyoeQ+tWyljcbks88CPu/hGhTWA8sDRThI3//6gSscUmAa+6+1LoPNctVug6dYrPYDwz5dHA1909AoireJri5VcIjd+f6Mi42vgddpbrVkMYDHVrB4tKXLd89w3a+TO3qyeLrYMdxt9KTwEeqVQwcd3n7cDb7v7zrO3Z9YnHA3+pQGy9zaxvZpkwuONfCNfrrPiws4D/7ujYsrT4htcZrluWQtfpEeBMM0uY2SHA6kzVQUeJewR+HzjW3TdkbR+SaTQ2sz0JDaLvd3BshX6HjwCnmFnPeLDSMcCfOjK22BHA3zyeDho6/roVum/Qzp+5XbrrrLsnzSwz2GE1MN3d36pgSP8InAG8aWZ/jrddTuiltR+hqLgQOK8CsQ0FHo4nmaoB7nH335vZS8ADZvYN4O/ASRWIDTPrBXyRltfmp5W4bmZ2L2GQzMHxcPtXAteS/zrNInRhnE/oxXVOBWK7DOgJzIl/v5munocCU80sSeieen45B/QsENvEfL/DeFDSB4C/EqrOvuPuqY6Mzd1vp3UbGXTwdaPwfaNdP3Ma7kNERIra1auhRESkBEoWIiJSlJKFiIgUpWQhIiJFKVmIiEhRShYinYCZTTSz31U6DpFClCxERKSoXfqhPJHtZWanAxcBPQiDtV0ArAZ+CRwGrAROcfdl8cNkNwO9CEM+TInnExgdbx9CeGgr87BUHzObCewDvAKcnhl6Q6TSVLIQKZGZfQr4GmFAxf0IN/qvA70JY1IdAMwlPHkMYUKa78fzRLyZtf3XwDR3/wzwOcLAcxBGC72EMLfKnoQnc0U6BZUsREr3BcLkSi/Fw2LUEwZnS7NtILm7gYfMrD8wwN3nxtvvBB6Mx9cyd38YwN03AcSv96fMGEPxsA0jgT+W/8cSKU7JQqR0CeBOd8+eSQ4z+/ec49qqOso3PHTG5qzlFPr7lE5E1VAipXsSONHMdoMwr7aZ7UH4OzoxPuY04I/uvhpYmTXxzRnA3HiegUYz+2r8Gj3jQRBFOjV9cxEpkbv/1cx+SJgtsApoBr4DrAc+bWavEBq7vxafchZwc5wM3mfb6J5nAL80s6nxa1RkpF6R7aFRZ0V2kpmtc/c+lY5DpJxUDSUiIkWpZCEiIkWpZCEiIkUpWYiISFFKFiIiUpSShYiIFKVkISIiRf1/V+kJpUnh5p8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1c4d561630>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "## Plotting the error with the number of iterations\n",
    "## With each iteration the error reduces smoothly\n",
    "plt.plot(history.history['acc'])\n",
    "plt.plot(history.history['val_acc'])\n",
    "plt.title('model accuracy')\n",
    "plt.ylabel('accuracy')\n",
    "plt.xlabel('epoch')\n",
    "plt.legend(['train', 'test'], loc='upper left')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 376,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "512/594 [========================>.....] - ETA: 0s"
     ]
    }
   ],
   "source": [
    "## read test file\n",
    "test = pd.read_csv('https://s3.amazonaws.com/hackerday.datascience/118/test.csv')\n",
    "index = test.pop('id')\n",
    "\n",
    "## we need to perform the same transformations from the training set to the test set\n",
    "test = preprocessing.MinMaxScaler().fit(test).transform(test)\n",
    "test = StandardScaler().fit(test).transform(test)\n",
    "yPred = model.predict_proba(test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 377,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "yPred = pd.DataFrame(yPred,index=index,columns=sort(parent_data.species.unique()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 378,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Acer_Capillipes</th>\n",
       "      <th>Acer_Circinatum</th>\n",
       "      <th>Acer_Mono</th>\n",
       "      <th>Acer_Opalus</th>\n",
       "      <th>Acer_Palmatum</th>\n",
       "      <th>Acer_Pictum</th>\n",
       "      <th>Acer_Platanoids</th>\n",
       "      <th>Acer_Rubrum</th>\n",
       "      <th>Acer_Rufinerve</th>\n",
       "      <th>Acer_Saccharinum</th>\n",
       "      <th>...</th>\n",
       "      <th>Salix_Fragilis</th>\n",
       "      <th>Salix_Intergra</th>\n",
       "      <th>Sorbus_Aria</th>\n",
       "      <th>Tilia_Oliveri</th>\n",
       "      <th>Tilia_Platyphyllos</th>\n",
       "      <th>Tilia_Tomentosa</th>\n",
       "      <th>Ulmus_Bergmanniana</th>\n",
       "      <th>Viburnum_Tinus</th>\n",
       "      <th>Viburnum_x_Rhytidophylloides</th>\n",
       "      <th>Zelkova_Serrata</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1.100567e-13</td>\n",
       "      <td>4.523121e-12</td>\n",
       "      <td>8.577789e-17</td>\n",
       "      <td>9.206923e-10</td>\n",
       "      <td>8.285942e-12</td>\n",
       "      <td>1.509020e-15</td>\n",
       "      <td>2.958439e-14</td>\n",
       "      <td>3.824437e-15</td>\n",
       "      <td>1.134421e-13</td>\n",
       "      <td>6.414147e-13</td>\n",
       "      <td>...</td>\n",
       "      <td>3.282422e-14</td>\n",
       "      <td>1.786060e-09</td>\n",
       "      <td>2.646354e-14</td>\n",
       "      <td>9.859845e-16</td>\n",
       "      <td>1.019394e-12</td>\n",
       "      <td>3.116100e-13</td>\n",
       "      <td>5.175158e-17</td>\n",
       "      <td>3.606400e-16</td>\n",
       "      <td>1.711030e-11</td>\n",
       "      <td>6.657540e-18</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>7.021929e-18</td>\n",
       "      <td>4.000996e-15</td>\n",
       "      <td>2.441575e-14</td>\n",
       "      <td>4.769028e-13</td>\n",
       "      <td>5.807748e-16</td>\n",
       "      <td>5.097615e-15</td>\n",
       "      <td>1.225808e-11</td>\n",
       "      <td>9.464514e-18</td>\n",
       "      <td>2.444483e-14</td>\n",
       "      <td>1.207861e-14</td>\n",
       "      <td>...</td>\n",
       "      <td>4.190795e-14</td>\n",
       "      <td>1.234934e-15</td>\n",
       "      <td>9.432256e-18</td>\n",
       "      <td>4.393622e-19</td>\n",
       "      <td>2.914992e-21</td>\n",
       "      <td>7.358572e-13</td>\n",
       "      <td>7.247904e-16</td>\n",
       "      <td>3.402543e-12</td>\n",
       "      <td>1.270131e-16</td>\n",
       "      <td>3.994680e-16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>6.582326e-10</td>\n",
       "      <td>9.985359e-01</td>\n",
       "      <td>9.144836e-12</td>\n",
       "      <td>2.812427e-11</td>\n",
       "      <td>1.459809e-03</td>\n",
       "      <td>2.409311e-07</td>\n",
       "      <td>8.945187e-12</td>\n",
       "      <td>6.738888e-09</td>\n",
       "      <td>8.271952e-08</td>\n",
       "      <td>3.784165e-08</td>\n",
       "      <td>...</td>\n",
       "      <td>3.697578e-11</td>\n",
       "      <td>7.241681e-13</td>\n",
       "      <td>3.333965e-11</td>\n",
       "      <td>1.411578e-13</td>\n",
       "      <td>2.276031e-15</td>\n",
       "      <td>3.934525e-12</td>\n",
       "      <td>3.051722e-13</td>\n",
       "      <td>4.054768e-17</td>\n",
       "      <td>1.650489e-11</td>\n",
       "      <td>8.774527e-09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>1.187742e-11</td>\n",
       "      <td>8.054749e-06</td>\n",
       "      <td>1.798010e-08</td>\n",
       "      <td>4.362862e-12</td>\n",
       "      <td>9.735817e-11</td>\n",
       "      <td>4.675951e-10</td>\n",
       "      <td>7.807548e-06</td>\n",
       "      <td>4.654334e-08</td>\n",
       "      <td>1.043819e-08</td>\n",
       "      <td>3.076497e-06</td>\n",
       "      <td>...</td>\n",
       "      <td>4.297972e-10</td>\n",
       "      <td>1.024196e-11</td>\n",
       "      <td>3.386231e-09</td>\n",
       "      <td>5.864905e-09</td>\n",
       "      <td>1.580573e-12</td>\n",
       "      <td>1.164487e-07</td>\n",
       "      <td>1.359063e-03</td>\n",
       "      <td>1.024508e-08</td>\n",
       "      <td>5.079954e-12</td>\n",
       "      <td>9.817265e-06</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>8.051978e-13</td>\n",
       "      <td>2.111511e-12</td>\n",
       "      <td>7.767755e-17</td>\n",
       "      <td>4.316340e-16</td>\n",
       "      <td>1.290511e-14</td>\n",
       "      <td>1.377246e-15</td>\n",
       "      <td>1.163136e-11</td>\n",
       "      <td>1.870398e-13</td>\n",
       "      <td>1.789570e-09</td>\n",
       "      <td>2.881625e-10</td>\n",
       "      <td>...</td>\n",
       "      <td>2.703658e-09</td>\n",
       "      <td>3.289086e-14</td>\n",
       "      <td>1.281434e-09</td>\n",
       "      <td>1.899380e-15</td>\n",
       "      <td>1.057438e-08</td>\n",
       "      <td>2.947929e-11</td>\n",
       "      <td>3.402397e-07</td>\n",
       "      <td>4.202475e-10</td>\n",
       "      <td>8.195366e-12</td>\n",
       "      <td>3.720808e-14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>2.825657e-08</td>\n",
       "      <td>6.422120e-14</td>\n",
       "      <td>2.344162e-11</td>\n",
       "      <td>9.966781e-01</td>\n",
       "      <td>2.436027e-15</td>\n",
       "      <td>5.843078e-15</td>\n",
       "      <td>3.982535e-12</td>\n",
       "      <td>1.302257e-10</td>\n",
       "      <td>3.335089e-11</td>\n",
       "      <td>1.306973e-16</td>\n",
       "      <td>...</td>\n",
       "      <td>8.187264e-08</td>\n",
       "      <td>6.993604e-09</td>\n",
       "      <td>9.171115e-07</td>\n",
       "      <td>3.584030e-12</td>\n",
       "      <td>1.666134e-09</td>\n",
       "      <td>9.215131e-08</td>\n",
       "      <td>2.484089e-10</td>\n",
       "      <td>5.835687e-10</td>\n",
       "      <td>5.496554e-06</td>\n",
       "      <td>3.787366e-09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>2.033156e-11</td>\n",
       "      <td>4.091485e-13</td>\n",
       "      <td>8.987290e-13</td>\n",
       "      <td>1.000000e+00</td>\n",
       "      <td>5.168866e-17</td>\n",
       "      <td>6.239816e-17</td>\n",
       "      <td>1.459896e-14</td>\n",
       "      <td>1.184547e-13</td>\n",
       "      <td>3.144879e-14</td>\n",
       "      <td>1.300444e-17</td>\n",
       "      <td>...</td>\n",
       "      <td>1.006810e-11</td>\n",
       "      <td>5.945090e-12</td>\n",
       "      <td>3.358122e-11</td>\n",
       "      <td>1.334341e-14</td>\n",
       "      <td>3.279488e-14</td>\n",
       "      <td>1.488782e-09</td>\n",
       "      <td>1.030237e-15</td>\n",
       "      <td>4.863621e-15</td>\n",
       "      <td>4.026451e-11</td>\n",
       "      <td>7.848215e-10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>1.033839e-15</td>\n",
       "      <td>1.296051e-09</td>\n",
       "      <td>3.457072e-13</td>\n",
       "      <td>2.803559e-10</td>\n",
       "      <td>2.212990e-13</td>\n",
       "      <td>2.522819e-07</td>\n",
       "      <td>4.833309e-10</td>\n",
       "      <td>1.047781e-11</td>\n",
       "      <td>4.771951e-15</td>\n",
       "      <td>4.507790e-12</td>\n",
       "      <td>...</td>\n",
       "      <td>4.628704e-13</td>\n",
       "      <td>1.314124e-14</td>\n",
       "      <td>8.438073e-13</td>\n",
       "      <td>3.717702e-12</td>\n",
       "      <td>1.499137e-11</td>\n",
       "      <td>1.288701e-15</td>\n",
       "      <td>1.544091e-17</td>\n",
       "      <td>1.991993e-16</td>\n",
       "      <td>8.668355e-15</td>\n",
       "      <td>1.697510e-12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>1.100359e-11</td>\n",
       "      <td>1.704271e-13</td>\n",
       "      <td>5.128198e-15</td>\n",
       "      <td>1.090811e-15</td>\n",
       "      <td>2.639401e-12</td>\n",
       "      <td>6.217043e-13</td>\n",
       "      <td>1.299702e-14</td>\n",
       "      <td>1.085316e-14</td>\n",
       "      <td>1.165164e-08</td>\n",
       "      <td>2.907105e-15</td>\n",
       "      <td>...</td>\n",
       "      <td>4.001051e-14</td>\n",
       "      <td>7.426835e-16</td>\n",
       "      <td>4.766542e-18</td>\n",
       "      <td>6.818104e-17</td>\n",
       "      <td>7.135447e-18</td>\n",
       "      <td>7.726076e-12</td>\n",
       "      <td>4.496066e-18</td>\n",
       "      <td>4.528057e-21</td>\n",
       "      <td>1.178677e-18</td>\n",
       "      <td>7.821341e-19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>7.381905e-10</td>\n",
       "      <td>3.890239e-12</td>\n",
       "      <td>2.000091e-13</td>\n",
       "      <td>5.556615e-14</td>\n",
       "      <td>3.533821e-13</td>\n",
       "      <td>6.556269e-13</td>\n",
       "      <td>1.846105e-10</td>\n",
       "      <td>4.806659e-10</td>\n",
       "      <td>1.000000e+00</td>\n",
       "      <td>5.259774e-14</td>\n",
       "      <td>...</td>\n",
       "      <td>4.879527e-12</td>\n",
       "      <td>3.789398e-14</td>\n",
       "      <td>6.663632e-13</td>\n",
       "      <td>3.770735e-16</td>\n",
       "      <td>4.004558e-13</td>\n",
       "      <td>2.315102e-09</td>\n",
       "      <td>1.549911e-13</td>\n",
       "      <td>3.163423e-20</td>\n",
       "      <td>1.284225e-17</td>\n",
       "      <td>7.948440e-14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>1.995443e-11</td>\n",
       "      <td>1.960460e-15</td>\n",
       "      <td>2.950365e-13</td>\n",
       "      <td>3.091986e-09</td>\n",
       "      <td>1.027909e-15</td>\n",
       "      <td>8.098866e-15</td>\n",
       "      <td>1.033650e-14</td>\n",
       "      <td>3.820859e-14</td>\n",
       "      <td>7.545042e-19</td>\n",
       "      <td>1.943238e-15</td>\n",
       "      <td>...</td>\n",
       "      <td>7.354428e-10</td>\n",
       "      <td>9.904720e-13</td>\n",
       "      <td>6.942738e-11</td>\n",
       "      <td>6.925548e-10</td>\n",
       "      <td>1.321778e-12</td>\n",
       "      <td>1.007988e-17</td>\n",
       "      <td>3.658791e-10</td>\n",
       "      <td>2.020209e-08</td>\n",
       "      <td>2.916461e-08</td>\n",
       "      <td>1.007804e-11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>36</th>\n",
       "      <td>1.970313e-10</td>\n",
       "      <td>1.132217e-10</td>\n",
       "      <td>1.241150e-13</td>\n",
       "      <td>1.122959e-14</td>\n",
       "      <td>1.863013e-09</td>\n",
       "      <td>9.399079e-11</td>\n",
       "      <td>1.447345e-13</td>\n",
       "      <td>9.701109e-13</td>\n",
       "      <td>1.652875e-10</td>\n",
       "      <td>6.225125e-08</td>\n",
       "      <td>...</td>\n",
       "      <td>1.222793e-14</td>\n",
       "      <td>9.469657e-14</td>\n",
       "      <td>1.211937e-11</td>\n",
       "      <td>4.553429e-14</td>\n",
       "      <td>1.502922e-11</td>\n",
       "      <td>2.505662e-13</td>\n",
       "      <td>7.972182e-14</td>\n",
       "      <td>6.035194e-13</td>\n",
       "      <td>6.344118e-11</td>\n",
       "      <td>1.176658e-10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>6.598561e-12</td>\n",
       "      <td>2.784868e-14</td>\n",
       "      <td>1.220091e-13</td>\n",
       "      <td>1.302948e-14</td>\n",
       "      <td>1.797673e-13</td>\n",
       "      <td>2.958739e-13</td>\n",
       "      <td>1.655165e-18</td>\n",
       "      <td>9.447665e-18</td>\n",
       "      <td>1.108103e-15</td>\n",
       "      <td>4.870583e-11</td>\n",
       "      <td>...</td>\n",
       "      <td>9.840573e-15</td>\n",
       "      <td>1.152394e-12</td>\n",
       "      <td>4.302132e-21</td>\n",
       "      <td>2.938895e-14</td>\n",
       "      <td>1.298454e-20</td>\n",
       "      <td>3.049014e-15</td>\n",
       "      <td>3.376430e-16</td>\n",
       "      <td>1.042771e-14</td>\n",
       "      <td>4.552028e-12</td>\n",
       "      <td>7.101946e-15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41</th>\n",
       "      <td>1.465208e-12</td>\n",
       "      <td>8.789548e-14</td>\n",
       "      <td>1.481774e-10</td>\n",
       "      <td>1.439193e-11</td>\n",
       "      <td>2.832621e-15</td>\n",
       "      <td>1.293897e-13</td>\n",
       "      <td>1.226553e-15</td>\n",
       "      <td>3.497204e-12</td>\n",
       "      <td>1.684932e-13</td>\n",
       "      <td>2.279846e-15</td>\n",
       "      <td>...</td>\n",
       "      <td>1.326196e-14</td>\n",
       "      <td>7.139106e-11</td>\n",
       "      <td>2.020089e-12</td>\n",
       "      <td>1.091435e-08</td>\n",
       "      <td>3.809757e-09</td>\n",
       "      <td>2.686263e-08</td>\n",
       "      <td>1.495722e-14</td>\n",
       "      <td>1.710947e-17</td>\n",
       "      <td>1.296577e-14</td>\n",
       "      <td>1.204350e-11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>44</th>\n",
       "      <td>2.003853e-16</td>\n",
       "      <td>1.456806e-17</td>\n",
       "      <td>1.087141e-19</td>\n",
       "      <td>2.393618e-12</td>\n",
       "      <td>9.223323e-18</td>\n",
       "      <td>8.612346e-18</td>\n",
       "      <td>2.114608e-14</td>\n",
       "      <td>7.225575e-15</td>\n",
       "      <td>4.483392e-18</td>\n",
       "      <td>8.216112e-17</td>\n",
       "      <td>...</td>\n",
       "      <td>5.083104e-17</td>\n",
       "      <td>1.843578e-18</td>\n",
       "      <td>5.924906e-12</td>\n",
       "      <td>7.274407e-10</td>\n",
       "      <td>5.393786e-11</td>\n",
       "      <td>7.457406e-17</td>\n",
       "      <td>5.227025e-16</td>\n",
       "      <td>5.447494e-12</td>\n",
       "      <td>1.261569e-17</td>\n",
       "      <td>2.384301e-11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>46</th>\n",
       "      <td>1.056924e-11</td>\n",
       "      <td>1.072213e-11</td>\n",
       "      <td>1.532330e-12</td>\n",
       "      <td>4.063161e-09</td>\n",
       "      <td>3.883063e-11</td>\n",
       "      <td>5.195159e-12</td>\n",
       "      <td>7.588323e-12</td>\n",
       "      <td>2.170023e-12</td>\n",
       "      <td>1.337457e-14</td>\n",
       "      <td>1.800371e-12</td>\n",
       "      <td>...</td>\n",
       "      <td>1.473498e-16</td>\n",
       "      <td>1.590141e-15</td>\n",
       "      <td>2.095702e-08</td>\n",
       "      <td>2.907207e-09</td>\n",
       "      <td>7.410215e-10</td>\n",
       "      <td>4.677920e-12</td>\n",
       "      <td>3.981257e-14</td>\n",
       "      <td>7.492120e-13</td>\n",
       "      <td>2.962433e-09</td>\n",
       "      <td>1.783887e-10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>47</th>\n",
       "      <td>8.519352e-13</td>\n",
       "      <td>4.422726e-16</td>\n",
       "      <td>2.385852e-11</td>\n",
       "      <td>2.891780e-11</td>\n",
       "      <td>1.085416e-13</td>\n",
       "      <td>1.701831e-13</td>\n",
       "      <td>3.008384e-07</td>\n",
       "      <td>1.663468e-12</td>\n",
       "      <td>9.094730e-10</td>\n",
       "      <td>8.343822e-17</td>\n",
       "      <td>...</td>\n",
       "      <td>9.602602e-13</td>\n",
       "      <td>1.263053e-11</td>\n",
       "      <td>1.234521e-12</td>\n",
       "      <td>9.997999e-11</td>\n",
       "      <td>1.058491e-12</td>\n",
       "      <td>4.957739e-11</td>\n",
       "      <td>1.489013e-14</td>\n",
       "      <td>9.819394e-17</td>\n",
       "      <td>8.209071e-18</td>\n",
       "      <td>1.219728e-17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>51</th>\n",
       "      <td>6.538090e-12</td>\n",
       "      <td>2.107987e-14</td>\n",
       "      <td>2.329170e-08</td>\n",
       "      <td>1.921107e-13</td>\n",
       "      <td>3.183809e-13</td>\n",
       "      <td>1.152952e-13</td>\n",
       "      <td>2.542121e-15</td>\n",
       "      <td>3.022574e-09</td>\n",
       "      <td>8.922184e-16</td>\n",
       "      <td>1.683992e-16</td>\n",
       "      <td>...</td>\n",
       "      <td>2.108052e-14</td>\n",
       "      <td>5.232031e-09</td>\n",
       "      <td>3.388689e-16</td>\n",
       "      <td>1.007346e-10</td>\n",
       "      <td>2.347698e-15</td>\n",
       "      <td>2.945607e-14</td>\n",
       "      <td>2.081311e-15</td>\n",
       "      <td>2.070722e-15</td>\n",
       "      <td>4.766349e-15</td>\n",
       "      <td>5.047050e-13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>52</th>\n",
       "      <td>2.267130e-13</td>\n",
       "      <td>1.573555e-13</td>\n",
       "      <td>2.321346e-16</td>\n",
       "      <td>7.893316e-16</td>\n",
       "      <td>1.763005e-16</td>\n",
       "      <td>5.537398e-15</td>\n",
       "      <td>8.623029e-14</td>\n",
       "      <td>3.845027e-11</td>\n",
       "      <td>6.411569e-16</td>\n",
       "      <td>3.162061e-11</td>\n",
       "      <td>...</td>\n",
       "      <td>5.857743e-16</td>\n",
       "      <td>6.442187e-17</td>\n",
       "      <td>2.873894e-11</td>\n",
       "      <td>1.654784e-09</td>\n",
       "      <td>1.842315e-11</td>\n",
       "      <td>8.065446e-15</td>\n",
       "      <td>2.454948e-11</td>\n",
       "      <td>3.591680e-14</td>\n",
       "      <td>3.025734e-17</td>\n",
       "      <td>1.108393e-10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>53</th>\n",
       "      <td>1.996484e-11</td>\n",
       "      <td>3.316608e-11</td>\n",
       "      <td>4.424255e-17</td>\n",
       "      <td>4.488762e-13</td>\n",
       "      <td>1.006793e-10</td>\n",
       "      <td>2.320743e-10</td>\n",
       "      <td>3.367687e-11</td>\n",
       "      <td>1.274427e-12</td>\n",
       "      <td>3.423006e-14</td>\n",
       "      <td>1.081273e-06</td>\n",
       "      <td>...</td>\n",
       "      <td>1.205907e-10</td>\n",
       "      <td>1.489202e-13</td>\n",
       "      <td>4.119619e-11</td>\n",
       "      <td>3.507837e-11</td>\n",
       "      <td>1.614827e-10</td>\n",
       "      <td>9.897714e-18</td>\n",
       "      <td>1.958945e-14</td>\n",
       "      <td>9.139941e-16</td>\n",
       "      <td>9.664176e-14</td>\n",
       "      <td>7.140145e-15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>57</th>\n",
       "      <td>2.569280e-16</td>\n",
       "      <td>9.535493e-21</td>\n",
       "      <td>1.172866e-16</td>\n",
       "      <td>2.890266e-21</td>\n",
       "      <td>1.082019e-18</td>\n",
       "      <td>5.596091e-16</td>\n",
       "      <td>2.777931e-21</td>\n",
       "      <td>6.733954e-21</td>\n",
       "      <td>2.119615e-18</td>\n",
       "      <td>2.979319e-13</td>\n",
       "      <td>...</td>\n",
       "      <td>5.452628e-16</td>\n",
       "      <td>4.432358e-11</td>\n",
       "      <td>8.241710e-21</td>\n",
       "      <td>6.863271e-18</td>\n",
       "      <td>2.592655e-15</td>\n",
       "      <td>1.803881e-20</td>\n",
       "      <td>1.636640e-15</td>\n",
       "      <td>1.266608e-09</td>\n",
       "      <td>9.859725e-18</td>\n",
       "      <td>1.173085e-16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59</th>\n",
       "      <td>2.135648e-12</td>\n",
       "      <td>3.246070e-19</td>\n",
       "      <td>6.842772e-09</td>\n",
       "      <td>9.680691e-14</td>\n",
       "      <td>1.948529e-15</td>\n",
       "      <td>6.429568e-15</td>\n",
       "      <td>5.454058e-16</td>\n",
       "      <td>4.358833e-16</td>\n",
       "      <td>6.673993e-16</td>\n",
       "      <td>8.662914e-14</td>\n",
       "      <td>...</td>\n",
       "      <td>2.643999e-15</td>\n",
       "      <td>8.163256e-10</td>\n",
       "      <td>5.632342e-16</td>\n",
       "      <td>2.280415e-10</td>\n",
       "      <td>1.709218e-12</td>\n",
       "      <td>5.081167e-14</td>\n",
       "      <td>1.657441e-14</td>\n",
       "      <td>1.381596e-07</td>\n",
       "      <td>3.413611e-15</td>\n",
       "      <td>2.701047e-08</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>62</th>\n",
       "      <td>1.810889e-17</td>\n",
       "      <td>9.673627e-19</td>\n",
       "      <td>4.700603e-16</td>\n",
       "      <td>6.620722e-17</td>\n",
       "      <td>4.815472e-16</td>\n",
       "      <td>5.878153e-18</td>\n",
       "      <td>8.272032e-16</td>\n",
       "      <td>2.316106e-12</td>\n",
       "      <td>7.108221e-19</td>\n",
       "      <td>5.751609e-18</td>\n",
       "      <td>...</td>\n",
       "      <td>1.445401e-12</td>\n",
       "      <td>7.675778e-17</td>\n",
       "      <td>7.650751e-18</td>\n",
       "      <td>7.496285e-16</td>\n",
       "      <td>1.270634e-17</td>\n",
       "      <td>3.613535e-16</td>\n",
       "      <td>3.961770e-13</td>\n",
       "      <td>3.084394e-13</td>\n",
       "      <td>5.205927e-18</td>\n",
       "      <td>9.840851e-20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>65</th>\n",
       "      <td>5.411542e-18</td>\n",
       "      <td>8.114364e-18</td>\n",
       "      <td>3.076509e-13</td>\n",
       "      <td>2.399794e-16</td>\n",
       "      <td>6.677407e-14</td>\n",
       "      <td>1.532612e-14</td>\n",
       "      <td>5.760338e-14</td>\n",
       "      <td>2.290289e-19</td>\n",
       "      <td>3.091727e-13</td>\n",
       "      <td>2.775537e-17</td>\n",
       "      <td>...</td>\n",
       "      <td>1.133863e-13</td>\n",
       "      <td>5.230488e-15</td>\n",
       "      <td>5.537322e-16</td>\n",
       "      <td>6.550308e-20</td>\n",
       "      <td>5.939601e-13</td>\n",
       "      <td>1.671860e-11</td>\n",
       "      <td>9.104220e-08</td>\n",
       "      <td>9.503141e-07</td>\n",
       "      <td>2.706464e-07</td>\n",
       "      <td>6.524240e-15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>68</th>\n",
       "      <td>1.948618e-14</td>\n",
       "      <td>9.999999e-01</td>\n",
       "      <td>1.172414e-16</td>\n",
       "      <td>8.812521e-16</td>\n",
       "      <td>7.375148e-10</td>\n",
       "      <td>1.388137e-11</td>\n",
       "      <td>2.092402e-13</td>\n",
       "      <td>3.637857e-12</td>\n",
       "      <td>5.113902e-13</td>\n",
       "      <td>8.486563e-09</td>\n",
       "      <td>...</td>\n",
       "      <td>3.893287e-13</td>\n",
       "      <td>3.117775e-14</td>\n",
       "      <td>1.356166e-13</td>\n",
       "      <td>1.399994e-15</td>\n",
       "      <td>1.504449e-17</td>\n",
       "      <td>2.273274e-13</td>\n",
       "      <td>7.045400e-12</td>\n",
       "      <td>8.291578e-18</td>\n",
       "      <td>2.648222e-15</td>\n",
       "      <td>2.151844e-11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>70</th>\n",
       "      <td>5.269627e-17</td>\n",
       "      <td>4.372523e-18</td>\n",
       "      <td>4.005835e-16</td>\n",
       "      <td>5.391663e-20</td>\n",
       "      <td>2.448766e-15</td>\n",
       "      <td>6.968597e-15</td>\n",
       "      <td>4.707815e-17</td>\n",
       "      <td>1.716720e-14</td>\n",
       "      <td>2.894774e-17</td>\n",
       "      <td>3.463853e-16</td>\n",
       "      <td>...</td>\n",
       "      <td>1.758172e-21</td>\n",
       "      <td>3.971476e-17</td>\n",
       "      <td>4.130614e-16</td>\n",
       "      <td>1.308576e-12</td>\n",
       "      <td>4.075517e-10</td>\n",
       "      <td>5.281668e-22</td>\n",
       "      <td>4.188845e-18</td>\n",
       "      <td>4.931130e-14</td>\n",
       "      <td>1.854321e-17</td>\n",
       "      <td>5.566117e-15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>74</th>\n",
       "      <td>1.099731e-17</td>\n",
       "      <td>1.104746e-11</td>\n",
       "      <td>4.625400e-16</td>\n",
       "      <td>8.726854e-12</td>\n",
       "      <td>5.541854e-14</td>\n",
       "      <td>1.599964e-11</td>\n",
       "      <td>7.258569e-10</td>\n",
       "      <td>4.827092e-11</td>\n",
       "      <td>2.816854e-18</td>\n",
       "      <td>1.762910e-11</td>\n",
       "      <td>...</td>\n",
       "      <td>3.350263e-14</td>\n",
       "      <td>3.031792e-16</td>\n",
       "      <td>8.544992e-13</td>\n",
       "      <td>1.578178e-07</td>\n",
       "      <td>2.008114e-11</td>\n",
       "      <td>6.246726e-17</td>\n",
       "      <td>5.397668e-15</td>\n",
       "      <td>5.972485e-12</td>\n",
       "      <td>3.108843e-16</td>\n",
       "      <td>5.225053e-13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>77</th>\n",
       "      <td>3.102186e-22</td>\n",
       "      <td>1.413645e-16</td>\n",
       "      <td>1.641563e-20</td>\n",
       "      <td>2.508030e-17</td>\n",
       "      <td>4.044046e-18</td>\n",
       "      <td>1.665211e-16</td>\n",
       "      <td>5.137103e-16</td>\n",
       "      <td>3.886059e-14</td>\n",
       "      <td>8.999081e-19</td>\n",
       "      <td>1.481264e-20</td>\n",
       "      <td>...</td>\n",
       "      <td>1.457038e-19</td>\n",
       "      <td>8.205594e-23</td>\n",
       "      <td>2.039534e-18</td>\n",
       "      <td>7.772508e-18</td>\n",
       "      <td>5.523367e-16</td>\n",
       "      <td>8.080799e-19</td>\n",
       "      <td>2.196841e-17</td>\n",
       "      <td>2.236530e-19</td>\n",
       "      <td>1.460225e-20</td>\n",
       "      <td>8.232171e-18</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>79</th>\n",
       "      <td>7.593295e-11</td>\n",
       "      <td>3.252799e-11</td>\n",
       "      <td>1.223826e-09</td>\n",
       "      <td>2.573993e-13</td>\n",
       "      <td>1.726295e-10</td>\n",
       "      <td>2.966964e-08</td>\n",
       "      <td>3.153949e-14</td>\n",
       "      <td>4.096261e-14</td>\n",
       "      <td>1.259602e-10</td>\n",
       "      <td>5.589789e-11</td>\n",
       "      <td>...</td>\n",
       "      <td>6.556262e-15</td>\n",
       "      <td>1.822984e-12</td>\n",
       "      <td>7.178578e-12</td>\n",
       "      <td>5.613468e-14</td>\n",
       "      <td>1.274506e-12</td>\n",
       "      <td>2.791820e-18</td>\n",
       "      <td>6.579154e-17</td>\n",
       "      <td>2.351152e-20</td>\n",
       "      <td>4.227329e-11</td>\n",
       "      <td>3.455591e-12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>86</th>\n",
       "      <td>3.508014e-16</td>\n",
       "      <td>8.600095e-19</td>\n",
       "      <td>3.472731e-16</td>\n",
       "      <td>8.682136e-20</td>\n",
       "      <td>1.870421e-15</td>\n",
       "      <td>2.384315e-15</td>\n",
       "      <td>2.873639e-17</td>\n",
       "      <td>5.048463e-14</td>\n",
       "      <td>4.418808e-16</td>\n",
       "      <td>4.708499e-16</td>\n",
       "      <td>...</td>\n",
       "      <td>6.547691e-22</td>\n",
       "      <td>2.072110e-18</td>\n",
       "      <td>4.115656e-16</td>\n",
       "      <td>9.829232e-14</td>\n",
       "      <td>2.893057e-10</td>\n",
       "      <td>1.896076e-21</td>\n",
       "      <td>1.213178e-18</td>\n",
       "      <td>1.182124e-14</td>\n",
       "      <td>8.128915e-17</td>\n",
       "      <td>4.756759e-15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1493</th>\n",
       "      <td>2.969301e-13</td>\n",
       "      <td>7.804084e-06</td>\n",
       "      <td>3.108516e-12</td>\n",
       "      <td>4.109036e-16</td>\n",
       "      <td>3.118771e-09</td>\n",
       "      <td>5.998356e-10</td>\n",
       "      <td>1.792040e-12</td>\n",
       "      <td>3.766919e-12</td>\n",
       "      <td>4.375181e-16</td>\n",
       "      <td>1.974604e-07</td>\n",
       "      <td>...</td>\n",
       "      <td>9.650557e-11</td>\n",
       "      <td>8.803610e-14</td>\n",
       "      <td>8.311849e-10</td>\n",
       "      <td>3.141827e-10</td>\n",
       "      <td>1.879045e-16</td>\n",
       "      <td>1.877295e-14</td>\n",
       "      <td>5.732262e-07</td>\n",
       "      <td>2.652025e-10</td>\n",
       "      <td>4.759859e-09</td>\n",
       "      <td>5.686904e-13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1495</th>\n",
       "      <td>1.812362e-15</td>\n",
       "      <td>3.292646e-16</td>\n",
       "      <td>3.222262e-11</td>\n",
       "      <td>3.896402e-15</td>\n",
       "      <td>1.845267e-16</td>\n",
       "      <td>1.317627e-15</td>\n",
       "      <td>5.098437e-12</td>\n",
       "      <td>4.831895e-11</td>\n",
       "      <td>1.842755e-17</td>\n",
       "      <td>1.250072e-15</td>\n",
       "      <td>...</td>\n",
       "      <td>2.001354e-16</td>\n",
       "      <td>3.742200e-14</td>\n",
       "      <td>1.473726e-15</td>\n",
       "      <td>9.999995e-01</td>\n",
       "      <td>3.060203e-13</td>\n",
       "      <td>4.335020e-14</td>\n",
       "      <td>6.507226e-14</td>\n",
       "      <td>7.962058e-15</td>\n",
       "      <td>1.817717e-19</td>\n",
       "      <td>6.565551e-13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1497</th>\n",
       "      <td>1.864947e-08</td>\n",
       "      <td>3.249162e-08</td>\n",
       "      <td>8.523473e-14</td>\n",
       "      <td>2.020563e-14</td>\n",
       "      <td>1.295920e-08</td>\n",
       "      <td>6.684516e-13</td>\n",
       "      <td>2.761734e-15</td>\n",
       "      <td>1.123131e-12</td>\n",
       "      <td>1.339661e-06</td>\n",
       "      <td>6.498862e-11</td>\n",
       "      <td>...</td>\n",
       "      <td>2.846031e-09</td>\n",
       "      <td>1.278415e-13</td>\n",
       "      <td>5.446264e-12</td>\n",
       "      <td>1.729507e-17</td>\n",
       "      <td>7.798865e-13</td>\n",
       "      <td>2.300294e-09</td>\n",
       "      <td>4.989239e-08</td>\n",
       "      <td>3.793966e-14</td>\n",
       "      <td>1.538821e-10</td>\n",
       "      <td>2.164320e-13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1498</th>\n",
       "      <td>4.102460e-13</td>\n",
       "      <td>9.296678e-15</td>\n",
       "      <td>9.897263e-11</td>\n",
       "      <td>1.552501e-09</td>\n",
       "      <td>1.526536e-15</td>\n",
       "      <td>5.405603e-15</td>\n",
       "      <td>3.213088e-09</td>\n",
       "      <td>1.661359e-11</td>\n",
       "      <td>5.562145e-14</td>\n",
       "      <td>7.201335e-16</td>\n",
       "      <td>...</td>\n",
       "      <td>1.641972e-11</td>\n",
       "      <td>4.748598e-10</td>\n",
       "      <td>6.355669e-11</td>\n",
       "      <td>1.040376e-07</td>\n",
       "      <td>1.474038e-10</td>\n",
       "      <td>1.508352e-10</td>\n",
       "      <td>1.148279e-08</td>\n",
       "      <td>1.213848e-07</td>\n",
       "      <td>4.699070e-14</td>\n",
       "      <td>1.531802e-10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1503</th>\n",
       "      <td>6.116918e-10</td>\n",
       "      <td>8.450384e-12</td>\n",
       "      <td>8.806422e-12</td>\n",
       "      <td>7.999324e-11</td>\n",
       "      <td>1.116510e-12</td>\n",
       "      <td>1.410245e-11</td>\n",
       "      <td>9.614145e-10</td>\n",
       "      <td>1.336419e-10</td>\n",
       "      <td>1.620470e-09</td>\n",
       "      <td>4.468410e-10</td>\n",
       "      <td>...</td>\n",
       "      <td>2.572836e-08</td>\n",
       "      <td>5.597641e-11</td>\n",
       "      <td>1.701876e-14</td>\n",
       "      <td>1.910040e-13</td>\n",
       "      <td>6.642413e-13</td>\n",
       "      <td>4.248125e-11</td>\n",
       "      <td>3.120527e-12</td>\n",
       "      <td>6.098750e-12</td>\n",
       "      <td>5.683337e-12</td>\n",
       "      <td>1.367342e-13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1510</th>\n",
       "      <td>2.936844e-12</td>\n",
       "      <td>8.278224e-12</td>\n",
       "      <td>3.993911e-17</td>\n",
       "      <td>1.386000e-16</td>\n",
       "      <td>3.702940e-14</td>\n",
       "      <td>2.982486e-14</td>\n",
       "      <td>9.750951e-12</td>\n",
       "      <td>1.060213e-13</td>\n",
       "      <td>4.500720e-10</td>\n",
       "      <td>3.483351e-10</td>\n",
       "      <td>...</td>\n",
       "      <td>1.439277e-09</td>\n",
       "      <td>3.343043e-14</td>\n",
       "      <td>1.957307e-09</td>\n",
       "      <td>1.425542e-14</td>\n",
       "      <td>5.656099e-09</td>\n",
       "      <td>5.365640e-12</td>\n",
       "      <td>2.224549e-06</td>\n",
       "      <td>7.796701e-10</td>\n",
       "      <td>3.186961e-11</td>\n",
       "      <td>1.012459e-13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1513</th>\n",
       "      <td>1.574164e-12</td>\n",
       "      <td>7.944004e-14</td>\n",
       "      <td>7.193192e-16</td>\n",
       "      <td>5.906056e-12</td>\n",
       "      <td>8.056851e-13</td>\n",
       "      <td>2.348806e-15</td>\n",
       "      <td>2.006326e-11</td>\n",
       "      <td>8.456555e-11</td>\n",
       "      <td>4.120018e-12</td>\n",
       "      <td>3.456792e-13</td>\n",
       "      <td>...</td>\n",
       "      <td>2.108182e-14</td>\n",
       "      <td>1.329988e-12</td>\n",
       "      <td>6.951537e-07</td>\n",
       "      <td>1.868677e-09</td>\n",
       "      <td>9.999987e-01</td>\n",
       "      <td>4.919524e-13</td>\n",
       "      <td>6.013689e-13</td>\n",
       "      <td>1.109799e-14</td>\n",
       "      <td>2.382722e-14</td>\n",
       "      <td>2.411386e-12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1517</th>\n",
       "      <td>1.672991e-17</td>\n",
       "      <td>1.661677e-16</td>\n",
       "      <td>2.159849e-16</td>\n",
       "      <td>1.443810e-18</td>\n",
       "      <td>2.224512e-15</td>\n",
       "      <td>6.711362e-13</td>\n",
       "      <td>6.418392e-10</td>\n",
       "      <td>3.620880e-14</td>\n",
       "      <td>7.169400e-15</td>\n",
       "      <td>2.597779e-11</td>\n",
       "      <td>...</td>\n",
       "      <td>2.101740e-18</td>\n",
       "      <td>5.993154e-17</td>\n",
       "      <td>2.025782e-14</td>\n",
       "      <td>2.661710e-13</td>\n",
       "      <td>3.120904e-11</td>\n",
       "      <td>1.857066e-18</td>\n",
       "      <td>1.127345e-16</td>\n",
       "      <td>7.404645e-19</td>\n",
       "      <td>2.312901e-19</td>\n",
       "      <td>3.494384e-20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1522</th>\n",
       "      <td>2.120450e-16</td>\n",
       "      <td>1.068362e-18</td>\n",
       "      <td>2.150747e-16</td>\n",
       "      <td>2.978716e-14</td>\n",
       "      <td>1.596038e-17</td>\n",
       "      <td>1.388123e-16</td>\n",
       "      <td>1.212378e-17</td>\n",
       "      <td>3.558639e-16</td>\n",
       "      <td>1.433388e-14</td>\n",
       "      <td>1.236275e-19</td>\n",
       "      <td>...</td>\n",
       "      <td>2.348517e-20</td>\n",
       "      <td>2.355303e-17</td>\n",
       "      <td>6.746580e-13</td>\n",
       "      <td>8.440748e-17</td>\n",
       "      <td>1.377012e-14</td>\n",
       "      <td>1.660151e-15</td>\n",
       "      <td>2.490201e-18</td>\n",
       "      <td>3.056813e-14</td>\n",
       "      <td>4.666743e-17</td>\n",
       "      <td>3.629625e-11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1526</th>\n",
       "      <td>1.863229e-14</td>\n",
       "      <td>1.607073e-15</td>\n",
       "      <td>4.284540e-15</td>\n",
       "      <td>9.985375e-13</td>\n",
       "      <td>1.962655e-13</td>\n",
       "      <td>4.808831e-14</td>\n",
       "      <td>1.420719e-13</td>\n",
       "      <td>5.419017e-15</td>\n",
       "      <td>6.992984e-16</td>\n",
       "      <td>6.569276e-12</td>\n",
       "      <td>...</td>\n",
       "      <td>1.539849e-18</td>\n",
       "      <td>1.066409e-15</td>\n",
       "      <td>7.201691e-14</td>\n",
       "      <td>5.221241e-10</td>\n",
       "      <td>5.033192e-11</td>\n",
       "      <td>2.532269e-14</td>\n",
       "      <td>2.151262e-17</td>\n",
       "      <td>7.692921e-14</td>\n",
       "      <td>9.881657e-14</td>\n",
       "      <td>2.048114e-14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1528</th>\n",
       "      <td>3.628975e-13</td>\n",
       "      <td>8.224798e-12</td>\n",
       "      <td>2.522489e-16</td>\n",
       "      <td>3.913576e-12</td>\n",
       "      <td>1.802541e-10</td>\n",
       "      <td>7.246328e-15</td>\n",
       "      <td>2.825274e-13</td>\n",
       "      <td>4.064719e-12</td>\n",
       "      <td>1.194386e-12</td>\n",
       "      <td>9.027739e-14</td>\n",
       "      <td>...</td>\n",
       "      <td>4.770052e-17</td>\n",
       "      <td>6.411056e-15</td>\n",
       "      <td>2.580127e-12</td>\n",
       "      <td>8.378771e-15</td>\n",
       "      <td>9.132811e-11</td>\n",
       "      <td>7.615978e-18</td>\n",
       "      <td>5.448072e-16</td>\n",
       "      <td>9.248255e-12</td>\n",
       "      <td>2.528343e-11</td>\n",
       "      <td>8.635667e-10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1533</th>\n",
       "      <td>3.545209e-17</td>\n",
       "      <td>2.468518e-15</td>\n",
       "      <td>1.218784e-14</td>\n",
       "      <td>3.302464e-13</td>\n",
       "      <td>2.733462e-16</td>\n",
       "      <td>2.686714e-16</td>\n",
       "      <td>4.793426e-15</td>\n",
       "      <td>1.319790e-17</td>\n",
       "      <td>2.912601e-14</td>\n",
       "      <td>2.768880e-17</td>\n",
       "      <td>...</td>\n",
       "      <td>6.297233e-16</td>\n",
       "      <td>3.300282e-15</td>\n",
       "      <td>2.694800e-18</td>\n",
       "      <td>1.835219e-20</td>\n",
       "      <td>6.126980e-23</td>\n",
       "      <td>5.895878e-12</td>\n",
       "      <td>2.427059e-16</td>\n",
       "      <td>9.567590e-14</td>\n",
       "      <td>1.550148e-16</td>\n",
       "      <td>1.927212e-15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1534</th>\n",
       "      <td>4.261509e-08</td>\n",
       "      <td>6.161639e-11</td>\n",
       "      <td>1.741168e-08</td>\n",
       "      <td>5.526355e-11</td>\n",
       "      <td>7.959667e-11</td>\n",
       "      <td>1.022662e-10</td>\n",
       "      <td>4.742389e-09</td>\n",
       "      <td>9.853441e-01</td>\n",
       "      <td>3.783699e-08</td>\n",
       "      <td>6.246741e-12</td>\n",
       "      <td>...</td>\n",
       "      <td>9.464543e-09</td>\n",
       "      <td>2.613066e-11</td>\n",
       "      <td>1.445642e-08</td>\n",
       "      <td>7.926327e-06</td>\n",
       "      <td>1.124311e-09</td>\n",
       "      <td>3.759798e-09</td>\n",
       "      <td>1.876625e-08</td>\n",
       "      <td>9.228703e-16</td>\n",
       "      <td>1.639717e-16</td>\n",
       "      <td>4.531851e-10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1535</th>\n",
       "      <td>4.527461e-13</td>\n",
       "      <td>1.214905e-11</td>\n",
       "      <td>6.236588e-14</td>\n",
       "      <td>7.952613e-13</td>\n",
       "      <td>2.314325e-13</td>\n",
       "      <td>2.160491e-12</td>\n",
       "      <td>9.049773e-11</td>\n",
       "      <td>2.883129e-11</td>\n",
       "      <td>7.973303e-12</td>\n",
       "      <td>6.689090e-12</td>\n",
       "      <td>...</td>\n",
       "      <td>2.292579e-11</td>\n",
       "      <td>1.288477e-13</td>\n",
       "      <td>2.326142e-16</td>\n",
       "      <td>1.835840e-14</td>\n",
       "      <td>1.950038e-14</td>\n",
       "      <td>1.083543e-11</td>\n",
       "      <td>6.624015e-14</td>\n",
       "      <td>7.345137e-13</td>\n",
       "      <td>1.160792e-13</td>\n",
       "      <td>5.068391e-16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1537</th>\n",
       "      <td>1.875076e-11</td>\n",
       "      <td>2.832985e-15</td>\n",
       "      <td>1.337870e-16</td>\n",
       "      <td>1.537787e-11</td>\n",
       "      <td>1.485289e-15</td>\n",
       "      <td>1.442735e-17</td>\n",
       "      <td>1.779247e-13</td>\n",
       "      <td>7.210697e-11</td>\n",
       "      <td>2.041079e-11</td>\n",
       "      <td>2.475668e-14</td>\n",
       "      <td>...</td>\n",
       "      <td>1.081720e-11</td>\n",
       "      <td>1.620822e-12</td>\n",
       "      <td>2.821214e-07</td>\n",
       "      <td>4.118737e-12</td>\n",
       "      <td>9.999986e-01</td>\n",
       "      <td>3.079923e-12</td>\n",
       "      <td>5.586273e-12</td>\n",
       "      <td>1.288266e-14</td>\n",
       "      <td>1.170287e-13</td>\n",
       "      <td>6.398837e-14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1540</th>\n",
       "      <td>1.000000e+00</td>\n",
       "      <td>2.848977e-13</td>\n",
       "      <td>4.582626e-11</td>\n",
       "      <td>2.556945e-13</td>\n",
       "      <td>3.697082e-13</td>\n",
       "      <td>1.279067e-13</td>\n",
       "      <td>3.034068e-17</td>\n",
       "      <td>2.285233e-10</td>\n",
       "      <td>3.022316e-08</td>\n",
       "      <td>5.609508e-10</td>\n",
       "      <td>...</td>\n",
       "      <td>5.398624e-11</td>\n",
       "      <td>6.524499e-12</td>\n",
       "      <td>6.390135e-15</td>\n",
       "      <td>1.482173e-15</td>\n",
       "      <td>9.467809e-16</td>\n",
       "      <td>4.458600e-15</td>\n",
       "      <td>9.155572e-16</td>\n",
       "      <td>2.815659e-19</td>\n",
       "      <td>1.020880e-14</td>\n",
       "      <td>2.257140e-11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1542</th>\n",
       "      <td>4.364395e-08</td>\n",
       "      <td>6.965944e-13</td>\n",
       "      <td>1.282176e-14</td>\n",
       "      <td>8.817553e-11</td>\n",
       "      <td>3.735300e-13</td>\n",
       "      <td>3.923277e-16</td>\n",
       "      <td>3.961641e-13</td>\n",
       "      <td>1.437474e-09</td>\n",
       "      <td>3.539006e-10</td>\n",
       "      <td>4.108887e-12</td>\n",
       "      <td>...</td>\n",
       "      <td>7.449153e-11</td>\n",
       "      <td>7.247094e-14</td>\n",
       "      <td>1.376556e-07</td>\n",
       "      <td>2.051661e-13</td>\n",
       "      <td>6.265986e-08</td>\n",
       "      <td>1.689670e-13</td>\n",
       "      <td>1.324510e-14</td>\n",
       "      <td>2.418313e-18</td>\n",
       "      <td>1.633271e-12</td>\n",
       "      <td>5.152118e-15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1546</th>\n",
       "      <td>3.928658e-11</td>\n",
       "      <td>2.914797e-14</td>\n",
       "      <td>1.037312e-11</td>\n",
       "      <td>2.008772e-15</td>\n",
       "      <td>6.310271e-11</td>\n",
       "      <td>1.484305e-12</td>\n",
       "      <td>1.296666e-11</td>\n",
       "      <td>1.150691e-10</td>\n",
       "      <td>7.326273e-11</td>\n",
       "      <td>6.324610e-11</td>\n",
       "      <td>...</td>\n",
       "      <td>2.236341e-17</td>\n",
       "      <td>1.138992e-16</td>\n",
       "      <td>1.135046e-11</td>\n",
       "      <td>8.421563e-10</td>\n",
       "      <td>4.079166e-13</td>\n",
       "      <td>1.716784e-16</td>\n",
       "      <td>5.598427e-17</td>\n",
       "      <td>4.307538e-13</td>\n",
       "      <td>6.100846e-16</td>\n",
       "      <td>9.073183e-10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1553</th>\n",
       "      <td>1.478288e-12</td>\n",
       "      <td>6.550143e-14</td>\n",
       "      <td>1.045883e-10</td>\n",
       "      <td>1.316245e-10</td>\n",
       "      <td>8.403027e-15</td>\n",
       "      <td>7.505634e-15</td>\n",
       "      <td>8.681681e-15</td>\n",
       "      <td>7.851650e-13</td>\n",
       "      <td>3.853022e-12</td>\n",
       "      <td>1.916791e-15</td>\n",
       "      <td>...</td>\n",
       "      <td>1.184221e-14</td>\n",
       "      <td>2.381800e-09</td>\n",
       "      <td>6.324612e-11</td>\n",
       "      <td>1.394373e-09</td>\n",
       "      <td>1.512711e-06</td>\n",
       "      <td>1.239112e-09</td>\n",
       "      <td>1.879229e-15</td>\n",
       "      <td>5.880101e-16</td>\n",
       "      <td>5.364567e-15</td>\n",
       "      <td>8.808430e-11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1558</th>\n",
       "      <td>2.370303e-12</td>\n",
       "      <td>6.571207e-16</td>\n",
       "      <td>8.706281e-20</td>\n",
       "      <td>2.738688e-11</td>\n",
       "      <td>5.355386e-18</td>\n",
       "      <td>3.660241e-19</td>\n",
       "      <td>6.026238e-16</td>\n",
       "      <td>3.085394e-13</td>\n",
       "      <td>1.070444e-15</td>\n",
       "      <td>4.759077e-17</td>\n",
       "      <td>...</td>\n",
       "      <td>1.673084e-13</td>\n",
       "      <td>7.104135e-17</td>\n",
       "      <td>1.000000e+00</td>\n",
       "      <td>1.338219e-16</td>\n",
       "      <td>1.703113e-10</td>\n",
       "      <td>2.024396e-18</td>\n",
       "      <td>1.101755e-15</td>\n",
       "      <td>3.428683e-18</td>\n",
       "      <td>1.001852e-10</td>\n",
       "      <td>7.753925e-14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1560</th>\n",
       "      <td>3.559986e-11</td>\n",
       "      <td>7.639238e-09</td>\n",
       "      <td>1.505970e-07</td>\n",
       "      <td>8.315831e-10</td>\n",
       "      <td>3.193941e-10</td>\n",
       "      <td>2.898528e-08</td>\n",
       "      <td>3.431521e-08</td>\n",
       "      <td>4.576059e-11</td>\n",
       "      <td>1.670159e-07</td>\n",
       "      <td>2.697992e-12</td>\n",
       "      <td>...</td>\n",
       "      <td>1.195347e-07</td>\n",
       "      <td>2.315470e-08</td>\n",
       "      <td>1.402375e-12</td>\n",
       "      <td>4.550361e-13</td>\n",
       "      <td>3.020845e-13</td>\n",
       "      <td>1.218311e-08</td>\n",
       "      <td>1.092077e-08</td>\n",
       "      <td>8.900152e-10</td>\n",
       "      <td>1.464883e-10</td>\n",
       "      <td>8.165475e-11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1564</th>\n",
       "      <td>1.510183e-16</td>\n",
       "      <td>1.337111e-16</td>\n",
       "      <td>6.140029e-14</td>\n",
       "      <td>4.399224e-11</td>\n",
       "      <td>8.375600e-18</td>\n",
       "      <td>1.381682e-18</td>\n",
       "      <td>5.029755e-12</td>\n",
       "      <td>3.527743e-15</td>\n",
       "      <td>4.609437e-17</td>\n",
       "      <td>1.743136e-17</td>\n",
       "      <td>...</td>\n",
       "      <td>2.056882e-15</td>\n",
       "      <td>1.582777e-11</td>\n",
       "      <td>1.105254e-14</td>\n",
       "      <td>2.483515e-09</td>\n",
       "      <td>4.030555e-14</td>\n",
       "      <td>8.887296e-13</td>\n",
       "      <td>1.339163e-13</td>\n",
       "      <td>3.329998e-09</td>\n",
       "      <td>3.538699e-16</td>\n",
       "      <td>6.236393e-12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1565</th>\n",
       "      <td>5.403262e-16</td>\n",
       "      <td>1.524063e-12</td>\n",
       "      <td>2.687993e-16</td>\n",
       "      <td>8.874490e-15</td>\n",
       "      <td>3.793040e-14</td>\n",
       "      <td>3.133273e-14</td>\n",
       "      <td>1.704343e-15</td>\n",
       "      <td>8.164375e-15</td>\n",
       "      <td>1.551542e-17</td>\n",
       "      <td>6.109090e-14</td>\n",
       "      <td>...</td>\n",
       "      <td>1.810568e-18</td>\n",
       "      <td>5.710899e-17</td>\n",
       "      <td>5.059225e-11</td>\n",
       "      <td>3.028270e-11</td>\n",
       "      <td>2.381538e-13</td>\n",
       "      <td>4.829297e-18</td>\n",
       "      <td>2.542893e-16</td>\n",
       "      <td>2.363927e-11</td>\n",
       "      <td>6.451919e-14</td>\n",
       "      <td>7.762921e-09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1567</th>\n",
       "      <td>2.313635e-09</td>\n",
       "      <td>1.381947e-15</td>\n",
       "      <td>6.323156e-15</td>\n",
       "      <td>7.182990e-10</td>\n",
       "      <td>1.315375e-14</td>\n",
       "      <td>1.165915e-15</td>\n",
       "      <td>2.344044e-16</td>\n",
       "      <td>3.242564e-14</td>\n",
       "      <td>4.078147e-15</td>\n",
       "      <td>5.156168e-14</td>\n",
       "      <td>...</td>\n",
       "      <td>3.744255e-10</td>\n",
       "      <td>5.500723e-10</td>\n",
       "      <td>2.742796e-06</td>\n",
       "      <td>1.978573e-15</td>\n",
       "      <td>8.122959e-09</td>\n",
       "      <td>1.838869e-19</td>\n",
       "      <td>2.249433e-14</td>\n",
       "      <td>2.059629e-15</td>\n",
       "      <td>5.725772e-07</td>\n",
       "      <td>2.335336e-14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1573</th>\n",
       "      <td>2.260403e-17</td>\n",
       "      <td>9.815133e-20</td>\n",
       "      <td>1.832879e-16</td>\n",
       "      <td>9.398817e-18</td>\n",
       "      <td>7.573717e-17</td>\n",
       "      <td>4.856460e-17</td>\n",
       "      <td>8.519021e-19</td>\n",
       "      <td>5.317106e-22</td>\n",
       "      <td>4.212556e-20</td>\n",
       "      <td>2.149205e-16</td>\n",
       "      <td>...</td>\n",
       "      <td>3.972538e-13</td>\n",
       "      <td>7.014879e-13</td>\n",
       "      <td>1.139381e-15</td>\n",
       "      <td>5.437242e-22</td>\n",
       "      <td>4.660121e-19</td>\n",
       "      <td>3.584236e-17</td>\n",
       "      <td>2.279172e-12</td>\n",
       "      <td>1.620015e-11</td>\n",
       "      <td>2.651923e-10</td>\n",
       "      <td>1.846844e-18</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1576</th>\n",
       "      <td>4.347881e-10</td>\n",
       "      <td>9.999977e-01</td>\n",
       "      <td>8.694265e-12</td>\n",
       "      <td>4.186338e-13</td>\n",
       "      <td>1.881923e-06</td>\n",
       "      <td>5.934739e-10</td>\n",
       "      <td>5.912785e-14</td>\n",
       "      <td>4.484356e-11</td>\n",
       "      <td>5.670179e-09</td>\n",
       "      <td>1.913602e-09</td>\n",
       "      <td>...</td>\n",
       "      <td>6.742487e-13</td>\n",
       "      <td>1.175636e-12</td>\n",
       "      <td>9.311637e-13</td>\n",
       "      <td>1.199286e-14</td>\n",
       "      <td>1.736303e-17</td>\n",
       "      <td>1.383317e-13</td>\n",
       "      <td>2.009885e-12</td>\n",
       "      <td>2.494133e-17</td>\n",
       "      <td>8.610624e-13</td>\n",
       "      <td>4.773474e-10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1577</th>\n",
       "      <td>4.140858e-11</td>\n",
       "      <td>1.313108e-12</td>\n",
       "      <td>5.029715e-14</td>\n",
       "      <td>2.149070e-10</td>\n",
       "      <td>2.407912e-15</td>\n",
       "      <td>1.017314e-14</td>\n",
       "      <td>7.860467e-12</td>\n",
       "      <td>1.398054e-07</td>\n",
       "      <td>7.196132e-08</td>\n",
       "      <td>2.739472e-16</td>\n",
       "      <td>...</td>\n",
       "      <td>7.478755e-12</td>\n",
       "      <td>1.560831e-14</td>\n",
       "      <td>1.995448e-07</td>\n",
       "      <td>7.830710e-14</td>\n",
       "      <td>5.373228e-09</td>\n",
       "      <td>1.231009e-07</td>\n",
       "      <td>1.272548e-10</td>\n",
       "      <td>3.576942e-15</td>\n",
       "      <td>5.204717e-15</td>\n",
       "      <td>4.233081e-09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1579</th>\n",
       "      <td>1.716466e-11</td>\n",
       "      <td>5.636857e-13</td>\n",
       "      <td>4.145411e-12</td>\n",
       "      <td>1.372693e-17</td>\n",
       "      <td>1.144304e-09</td>\n",
       "      <td>3.136543e-11</td>\n",
       "      <td>3.686264e-13</td>\n",
       "      <td>3.512956e-11</td>\n",
       "      <td>8.018702e-12</td>\n",
       "      <td>1.335887e-09</td>\n",
       "      <td>...</td>\n",
       "      <td>1.098607e-16</td>\n",
       "      <td>2.611521e-17</td>\n",
       "      <td>1.641580e-13</td>\n",
       "      <td>2.154722e-11</td>\n",
       "      <td>2.377172e-14</td>\n",
       "      <td>2.398065e-18</td>\n",
       "      <td>6.095616e-17</td>\n",
       "      <td>2.039829e-13</td>\n",
       "      <td>2.355133e-15</td>\n",
       "      <td>2.341982e-11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1580</th>\n",
       "      <td>4.149217e-12</td>\n",
       "      <td>1.576864e-12</td>\n",
       "      <td>3.303256e-10</td>\n",
       "      <td>5.424564e-10</td>\n",
       "      <td>7.235152e-13</td>\n",
       "      <td>4.934980e-13</td>\n",
       "      <td>1.494684e-11</td>\n",
       "      <td>4.450540e-10</td>\n",
       "      <td>8.959203e-13</td>\n",
       "      <td>5.354250e-15</td>\n",
       "      <td>...</td>\n",
       "      <td>6.928876e-11</td>\n",
       "      <td>2.335643e-08</td>\n",
       "      <td>2.371070e-13</td>\n",
       "      <td>4.621850e-09</td>\n",
       "      <td>6.074169e-14</td>\n",
       "      <td>3.857699e-11</td>\n",
       "      <td>1.777245e-13</td>\n",
       "      <td>1.733183e-14</td>\n",
       "      <td>3.682359e-13</td>\n",
       "      <td>2.326030e-13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1583</th>\n",
       "      <td>1.296508e-22</td>\n",
       "      <td>4.836796e-14</td>\n",
       "      <td>2.109411e-18</td>\n",
       "      <td>3.927947e-18</td>\n",
       "      <td>2.368114e-17</td>\n",
       "      <td>7.009047e-13</td>\n",
       "      <td>6.375283e-15</td>\n",
       "      <td>5.496598e-13</td>\n",
       "      <td>3.998001e-17</td>\n",
       "      <td>1.192353e-18</td>\n",
       "      <td>...</td>\n",
       "      <td>4.502414e-19</td>\n",
       "      <td>1.991471e-22</td>\n",
       "      <td>1.372112e-19</td>\n",
       "      <td>1.474226e-17</td>\n",
       "      <td>1.597324e-16</td>\n",
       "      <td>2.339374e-16</td>\n",
       "      <td>4.684897e-19</td>\n",
       "      <td>1.200517e-20</td>\n",
       "      <td>6.421883e-21</td>\n",
       "      <td>2.479388e-16</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>594 rows × 99 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      Acer_Capillipes  Acer_Circinatum     Acer_Mono   Acer_Opalus  \\\n",
       "id                                                                   \n",
       "4        1.100567e-13     4.523121e-12  8.577789e-17  9.206923e-10   \n",
       "7        7.021929e-18     4.000996e-15  2.441575e-14  4.769028e-13   \n",
       "9        6.582326e-10     9.985359e-01  9.144836e-12  2.812427e-11   \n",
       "12       1.187742e-11     8.054749e-06  1.798010e-08  4.362862e-12   \n",
       "13       8.051978e-13     2.111511e-12  7.767755e-17  4.316340e-16   \n",
       "16       2.825657e-08     6.422120e-14  2.344162e-11  9.966781e-01   \n",
       "19       2.033156e-11     4.091485e-13  8.987290e-13  1.000000e+00   \n",
       "23       1.033839e-15     1.296051e-09  3.457072e-13  2.803559e-10   \n",
       "24       1.100359e-11     1.704271e-13  5.128198e-15  1.090811e-15   \n",
       "28       7.381905e-10     3.890239e-12  2.000091e-13  5.556615e-14   \n",
       "33       1.995443e-11     1.960460e-15  2.950365e-13  3.091986e-09   \n",
       "36       1.970313e-10     1.132217e-10  1.241150e-13  1.122959e-14   \n",
       "39       6.598561e-12     2.784868e-14  1.220091e-13  1.302948e-14   \n",
       "41       1.465208e-12     8.789548e-14  1.481774e-10  1.439193e-11   \n",
       "44       2.003853e-16     1.456806e-17  1.087141e-19  2.393618e-12   \n",
       "46       1.056924e-11     1.072213e-11  1.532330e-12  4.063161e-09   \n",
       "47       8.519352e-13     4.422726e-16  2.385852e-11  2.891780e-11   \n",
       "51       6.538090e-12     2.107987e-14  2.329170e-08  1.921107e-13   \n",
       "52       2.267130e-13     1.573555e-13  2.321346e-16  7.893316e-16   \n",
       "53       1.996484e-11     3.316608e-11  4.424255e-17  4.488762e-13   \n",
       "57       2.569280e-16     9.535493e-21  1.172866e-16  2.890266e-21   \n",
       "59       2.135648e-12     3.246070e-19  6.842772e-09  9.680691e-14   \n",
       "62       1.810889e-17     9.673627e-19  4.700603e-16  6.620722e-17   \n",
       "65       5.411542e-18     8.114364e-18  3.076509e-13  2.399794e-16   \n",
       "68       1.948618e-14     9.999999e-01  1.172414e-16  8.812521e-16   \n",
       "70       5.269627e-17     4.372523e-18  4.005835e-16  5.391663e-20   \n",
       "74       1.099731e-17     1.104746e-11  4.625400e-16  8.726854e-12   \n",
       "77       3.102186e-22     1.413645e-16  1.641563e-20  2.508030e-17   \n",
       "79       7.593295e-11     3.252799e-11  1.223826e-09  2.573993e-13   \n",
       "86       3.508014e-16     8.600095e-19  3.472731e-16  8.682136e-20   \n",
       "...               ...              ...           ...           ...   \n",
       "1493     2.969301e-13     7.804084e-06  3.108516e-12  4.109036e-16   \n",
       "1495     1.812362e-15     3.292646e-16  3.222262e-11  3.896402e-15   \n",
       "1497     1.864947e-08     3.249162e-08  8.523473e-14  2.020563e-14   \n",
       "1498     4.102460e-13     9.296678e-15  9.897263e-11  1.552501e-09   \n",
       "1503     6.116918e-10     8.450384e-12  8.806422e-12  7.999324e-11   \n",
       "1510     2.936844e-12     8.278224e-12  3.993911e-17  1.386000e-16   \n",
       "1513     1.574164e-12     7.944004e-14  7.193192e-16  5.906056e-12   \n",
       "1517     1.672991e-17     1.661677e-16  2.159849e-16  1.443810e-18   \n",
       "1522     2.120450e-16     1.068362e-18  2.150747e-16  2.978716e-14   \n",
       "1526     1.863229e-14     1.607073e-15  4.284540e-15  9.985375e-13   \n",
       "1528     3.628975e-13     8.224798e-12  2.522489e-16  3.913576e-12   \n",
       "1533     3.545209e-17     2.468518e-15  1.218784e-14  3.302464e-13   \n",
       "1534     4.261509e-08     6.161639e-11  1.741168e-08  5.526355e-11   \n",
       "1535     4.527461e-13     1.214905e-11  6.236588e-14  7.952613e-13   \n",
       "1537     1.875076e-11     2.832985e-15  1.337870e-16  1.537787e-11   \n",
       "1540     1.000000e+00     2.848977e-13  4.582626e-11  2.556945e-13   \n",
       "1542     4.364395e-08     6.965944e-13  1.282176e-14  8.817553e-11   \n",
       "1546     3.928658e-11     2.914797e-14  1.037312e-11  2.008772e-15   \n",
       "1553     1.478288e-12     6.550143e-14  1.045883e-10  1.316245e-10   \n",
       "1558     2.370303e-12     6.571207e-16  8.706281e-20  2.738688e-11   \n",
       "1560     3.559986e-11     7.639238e-09  1.505970e-07  8.315831e-10   \n",
       "1564     1.510183e-16     1.337111e-16  6.140029e-14  4.399224e-11   \n",
       "1565     5.403262e-16     1.524063e-12  2.687993e-16  8.874490e-15   \n",
       "1567     2.313635e-09     1.381947e-15  6.323156e-15  7.182990e-10   \n",
       "1573     2.260403e-17     9.815133e-20  1.832879e-16  9.398817e-18   \n",
       "1576     4.347881e-10     9.999977e-01  8.694265e-12  4.186338e-13   \n",
       "1577     4.140858e-11     1.313108e-12  5.029715e-14  2.149070e-10   \n",
       "1579     1.716466e-11     5.636857e-13  4.145411e-12  1.372693e-17   \n",
       "1580     4.149217e-12     1.576864e-12  3.303256e-10  5.424564e-10   \n",
       "1583     1.296508e-22     4.836796e-14  2.109411e-18  3.927947e-18   \n",
       "\n",
       "      Acer_Palmatum   Acer_Pictum  Acer_Platanoids   Acer_Rubrum  \\\n",
       "id                                                                 \n",
       "4      8.285942e-12  1.509020e-15     2.958439e-14  3.824437e-15   \n",
       "7      5.807748e-16  5.097615e-15     1.225808e-11  9.464514e-18   \n",
       "9      1.459809e-03  2.409311e-07     8.945187e-12  6.738888e-09   \n",
       "12     9.735817e-11  4.675951e-10     7.807548e-06  4.654334e-08   \n",
       "13     1.290511e-14  1.377246e-15     1.163136e-11  1.870398e-13   \n",
       "16     2.436027e-15  5.843078e-15     3.982535e-12  1.302257e-10   \n",
       "19     5.168866e-17  6.239816e-17     1.459896e-14  1.184547e-13   \n",
       "23     2.212990e-13  2.522819e-07     4.833309e-10  1.047781e-11   \n",
       "24     2.639401e-12  6.217043e-13     1.299702e-14  1.085316e-14   \n",
       "28     3.533821e-13  6.556269e-13     1.846105e-10  4.806659e-10   \n",
       "33     1.027909e-15  8.098866e-15     1.033650e-14  3.820859e-14   \n",
       "36     1.863013e-09  9.399079e-11     1.447345e-13  9.701109e-13   \n",
       "39     1.797673e-13  2.958739e-13     1.655165e-18  9.447665e-18   \n",
       "41     2.832621e-15  1.293897e-13     1.226553e-15  3.497204e-12   \n",
       "44     9.223323e-18  8.612346e-18     2.114608e-14  7.225575e-15   \n",
       "46     3.883063e-11  5.195159e-12     7.588323e-12  2.170023e-12   \n",
       "47     1.085416e-13  1.701831e-13     3.008384e-07  1.663468e-12   \n",
       "51     3.183809e-13  1.152952e-13     2.542121e-15  3.022574e-09   \n",
       "52     1.763005e-16  5.537398e-15     8.623029e-14  3.845027e-11   \n",
       "53     1.006793e-10  2.320743e-10     3.367687e-11  1.274427e-12   \n",
       "57     1.082019e-18  5.596091e-16     2.777931e-21  6.733954e-21   \n",
       "59     1.948529e-15  6.429568e-15     5.454058e-16  4.358833e-16   \n",
       "62     4.815472e-16  5.878153e-18     8.272032e-16  2.316106e-12   \n",
       "65     6.677407e-14  1.532612e-14     5.760338e-14  2.290289e-19   \n",
       "68     7.375148e-10  1.388137e-11     2.092402e-13  3.637857e-12   \n",
       "70     2.448766e-15  6.968597e-15     4.707815e-17  1.716720e-14   \n",
       "74     5.541854e-14  1.599964e-11     7.258569e-10  4.827092e-11   \n",
       "77     4.044046e-18  1.665211e-16     5.137103e-16  3.886059e-14   \n",
       "79     1.726295e-10  2.966964e-08     3.153949e-14  4.096261e-14   \n",
       "86     1.870421e-15  2.384315e-15     2.873639e-17  5.048463e-14   \n",
       "...             ...           ...              ...           ...   \n",
       "1493   3.118771e-09  5.998356e-10     1.792040e-12  3.766919e-12   \n",
       "1495   1.845267e-16  1.317627e-15     5.098437e-12  4.831895e-11   \n",
       "1497   1.295920e-08  6.684516e-13     2.761734e-15  1.123131e-12   \n",
       "1498   1.526536e-15  5.405603e-15     3.213088e-09  1.661359e-11   \n",
       "1503   1.116510e-12  1.410245e-11     9.614145e-10  1.336419e-10   \n",
       "1510   3.702940e-14  2.982486e-14     9.750951e-12  1.060213e-13   \n",
       "1513   8.056851e-13  2.348806e-15     2.006326e-11  8.456555e-11   \n",
       "1517   2.224512e-15  6.711362e-13     6.418392e-10  3.620880e-14   \n",
       "1522   1.596038e-17  1.388123e-16     1.212378e-17  3.558639e-16   \n",
       "1526   1.962655e-13  4.808831e-14     1.420719e-13  5.419017e-15   \n",
       "1528   1.802541e-10  7.246328e-15     2.825274e-13  4.064719e-12   \n",
       "1533   2.733462e-16  2.686714e-16     4.793426e-15  1.319790e-17   \n",
       "1534   7.959667e-11  1.022662e-10     4.742389e-09  9.853441e-01   \n",
       "1535   2.314325e-13  2.160491e-12     9.049773e-11  2.883129e-11   \n",
       "1537   1.485289e-15  1.442735e-17     1.779247e-13  7.210697e-11   \n",
       "1540   3.697082e-13  1.279067e-13     3.034068e-17  2.285233e-10   \n",
       "1542   3.735300e-13  3.923277e-16     3.961641e-13  1.437474e-09   \n",
       "1546   6.310271e-11  1.484305e-12     1.296666e-11  1.150691e-10   \n",
       "1553   8.403027e-15  7.505634e-15     8.681681e-15  7.851650e-13   \n",
       "1558   5.355386e-18  3.660241e-19     6.026238e-16  3.085394e-13   \n",
       "1560   3.193941e-10  2.898528e-08     3.431521e-08  4.576059e-11   \n",
       "1564   8.375600e-18  1.381682e-18     5.029755e-12  3.527743e-15   \n",
       "1565   3.793040e-14  3.133273e-14     1.704343e-15  8.164375e-15   \n",
       "1567   1.315375e-14  1.165915e-15     2.344044e-16  3.242564e-14   \n",
       "1573   7.573717e-17  4.856460e-17     8.519021e-19  5.317106e-22   \n",
       "1576   1.881923e-06  5.934739e-10     5.912785e-14  4.484356e-11   \n",
       "1577   2.407912e-15  1.017314e-14     7.860467e-12  1.398054e-07   \n",
       "1579   1.144304e-09  3.136543e-11     3.686264e-13  3.512956e-11   \n",
       "1580   7.235152e-13  4.934980e-13     1.494684e-11  4.450540e-10   \n",
       "1583   2.368114e-17  7.009047e-13     6.375283e-15  5.496598e-13   \n",
       "\n",
       "      Acer_Rufinerve  Acer_Saccharinum       ...         Salix_Fragilis  \\\n",
       "id                                           ...                          \n",
       "4       1.134421e-13      6.414147e-13       ...           3.282422e-14   \n",
       "7       2.444483e-14      1.207861e-14       ...           4.190795e-14   \n",
       "9       8.271952e-08      3.784165e-08       ...           3.697578e-11   \n",
       "12      1.043819e-08      3.076497e-06       ...           4.297972e-10   \n",
       "13      1.789570e-09      2.881625e-10       ...           2.703658e-09   \n",
       "16      3.335089e-11      1.306973e-16       ...           8.187264e-08   \n",
       "19      3.144879e-14      1.300444e-17       ...           1.006810e-11   \n",
       "23      4.771951e-15      4.507790e-12       ...           4.628704e-13   \n",
       "24      1.165164e-08      2.907105e-15       ...           4.001051e-14   \n",
       "28      1.000000e+00      5.259774e-14       ...           4.879527e-12   \n",
       "33      7.545042e-19      1.943238e-15       ...           7.354428e-10   \n",
       "36      1.652875e-10      6.225125e-08       ...           1.222793e-14   \n",
       "39      1.108103e-15      4.870583e-11       ...           9.840573e-15   \n",
       "41      1.684932e-13      2.279846e-15       ...           1.326196e-14   \n",
       "44      4.483392e-18      8.216112e-17       ...           5.083104e-17   \n",
       "46      1.337457e-14      1.800371e-12       ...           1.473498e-16   \n",
       "47      9.094730e-10      8.343822e-17       ...           9.602602e-13   \n",
       "51      8.922184e-16      1.683992e-16       ...           2.108052e-14   \n",
       "52      6.411569e-16      3.162061e-11       ...           5.857743e-16   \n",
       "53      3.423006e-14      1.081273e-06       ...           1.205907e-10   \n",
       "57      2.119615e-18      2.979319e-13       ...           5.452628e-16   \n",
       "59      6.673993e-16      8.662914e-14       ...           2.643999e-15   \n",
       "62      7.108221e-19      5.751609e-18       ...           1.445401e-12   \n",
       "65      3.091727e-13      2.775537e-17       ...           1.133863e-13   \n",
       "68      5.113902e-13      8.486563e-09       ...           3.893287e-13   \n",
       "70      2.894774e-17      3.463853e-16       ...           1.758172e-21   \n",
       "74      2.816854e-18      1.762910e-11       ...           3.350263e-14   \n",
       "77      8.999081e-19      1.481264e-20       ...           1.457038e-19   \n",
       "79      1.259602e-10      5.589789e-11       ...           6.556262e-15   \n",
       "86      4.418808e-16      4.708499e-16       ...           6.547691e-22   \n",
       "...              ...               ...       ...                    ...   \n",
       "1493    4.375181e-16      1.974604e-07       ...           9.650557e-11   \n",
       "1495    1.842755e-17      1.250072e-15       ...           2.001354e-16   \n",
       "1497    1.339661e-06      6.498862e-11       ...           2.846031e-09   \n",
       "1498    5.562145e-14      7.201335e-16       ...           1.641972e-11   \n",
       "1503    1.620470e-09      4.468410e-10       ...           2.572836e-08   \n",
       "1510    4.500720e-10      3.483351e-10       ...           1.439277e-09   \n",
       "1513    4.120018e-12      3.456792e-13       ...           2.108182e-14   \n",
       "1517    7.169400e-15      2.597779e-11       ...           2.101740e-18   \n",
       "1522    1.433388e-14      1.236275e-19       ...           2.348517e-20   \n",
       "1526    6.992984e-16      6.569276e-12       ...           1.539849e-18   \n",
       "1528    1.194386e-12      9.027739e-14       ...           4.770052e-17   \n",
       "1533    2.912601e-14      2.768880e-17       ...           6.297233e-16   \n",
       "1534    3.783699e-08      6.246741e-12       ...           9.464543e-09   \n",
       "1535    7.973303e-12      6.689090e-12       ...           2.292579e-11   \n",
       "1537    2.041079e-11      2.475668e-14       ...           1.081720e-11   \n",
       "1540    3.022316e-08      5.609508e-10       ...           5.398624e-11   \n",
       "1542    3.539006e-10      4.108887e-12       ...           7.449153e-11   \n",
       "1546    7.326273e-11      6.324610e-11       ...           2.236341e-17   \n",
       "1553    3.853022e-12      1.916791e-15       ...           1.184221e-14   \n",
       "1558    1.070444e-15      4.759077e-17       ...           1.673084e-13   \n",
       "1560    1.670159e-07      2.697992e-12       ...           1.195347e-07   \n",
       "1564    4.609437e-17      1.743136e-17       ...           2.056882e-15   \n",
       "1565    1.551542e-17      6.109090e-14       ...           1.810568e-18   \n",
       "1567    4.078147e-15      5.156168e-14       ...           3.744255e-10   \n",
       "1573    4.212556e-20      2.149205e-16       ...           3.972538e-13   \n",
       "1576    5.670179e-09      1.913602e-09       ...           6.742487e-13   \n",
       "1577    7.196132e-08      2.739472e-16       ...           7.478755e-12   \n",
       "1579    8.018702e-12      1.335887e-09       ...           1.098607e-16   \n",
       "1580    8.959203e-13      5.354250e-15       ...           6.928876e-11   \n",
       "1583    3.998001e-17      1.192353e-18       ...           4.502414e-19   \n",
       "\n",
       "      Salix_Intergra   Sorbus_Aria  Tilia_Oliveri  Tilia_Platyphyllos  \\\n",
       "id                                                                      \n",
       "4       1.786060e-09  2.646354e-14   9.859845e-16        1.019394e-12   \n",
       "7       1.234934e-15  9.432256e-18   4.393622e-19        2.914992e-21   \n",
       "9       7.241681e-13  3.333965e-11   1.411578e-13        2.276031e-15   \n",
       "12      1.024196e-11  3.386231e-09   5.864905e-09        1.580573e-12   \n",
       "13      3.289086e-14  1.281434e-09   1.899380e-15        1.057438e-08   \n",
       "16      6.993604e-09  9.171115e-07   3.584030e-12        1.666134e-09   \n",
       "19      5.945090e-12  3.358122e-11   1.334341e-14        3.279488e-14   \n",
       "23      1.314124e-14  8.438073e-13   3.717702e-12        1.499137e-11   \n",
       "24      7.426835e-16  4.766542e-18   6.818104e-17        7.135447e-18   \n",
       "28      3.789398e-14  6.663632e-13   3.770735e-16        4.004558e-13   \n",
       "33      9.904720e-13  6.942738e-11   6.925548e-10        1.321778e-12   \n",
       "36      9.469657e-14  1.211937e-11   4.553429e-14        1.502922e-11   \n",
       "39      1.152394e-12  4.302132e-21   2.938895e-14        1.298454e-20   \n",
       "41      7.139106e-11  2.020089e-12   1.091435e-08        3.809757e-09   \n",
       "44      1.843578e-18  5.924906e-12   7.274407e-10        5.393786e-11   \n",
       "46      1.590141e-15  2.095702e-08   2.907207e-09        7.410215e-10   \n",
       "47      1.263053e-11  1.234521e-12   9.997999e-11        1.058491e-12   \n",
       "51      5.232031e-09  3.388689e-16   1.007346e-10        2.347698e-15   \n",
       "52      6.442187e-17  2.873894e-11   1.654784e-09        1.842315e-11   \n",
       "53      1.489202e-13  4.119619e-11   3.507837e-11        1.614827e-10   \n",
       "57      4.432358e-11  8.241710e-21   6.863271e-18        2.592655e-15   \n",
       "59      8.163256e-10  5.632342e-16   2.280415e-10        1.709218e-12   \n",
       "62      7.675778e-17  7.650751e-18   7.496285e-16        1.270634e-17   \n",
       "65      5.230488e-15  5.537322e-16   6.550308e-20        5.939601e-13   \n",
       "68      3.117775e-14  1.356166e-13   1.399994e-15        1.504449e-17   \n",
       "70      3.971476e-17  4.130614e-16   1.308576e-12        4.075517e-10   \n",
       "74      3.031792e-16  8.544992e-13   1.578178e-07        2.008114e-11   \n",
       "77      8.205594e-23  2.039534e-18   7.772508e-18        5.523367e-16   \n",
       "79      1.822984e-12  7.178578e-12   5.613468e-14        1.274506e-12   \n",
       "86      2.072110e-18  4.115656e-16   9.829232e-14        2.893057e-10   \n",
       "...              ...           ...            ...                 ...   \n",
       "1493    8.803610e-14  8.311849e-10   3.141827e-10        1.879045e-16   \n",
       "1495    3.742200e-14  1.473726e-15   9.999995e-01        3.060203e-13   \n",
       "1497    1.278415e-13  5.446264e-12   1.729507e-17        7.798865e-13   \n",
       "1498    4.748598e-10  6.355669e-11   1.040376e-07        1.474038e-10   \n",
       "1503    5.597641e-11  1.701876e-14   1.910040e-13        6.642413e-13   \n",
       "1510    3.343043e-14  1.957307e-09   1.425542e-14        5.656099e-09   \n",
       "1513    1.329988e-12  6.951537e-07   1.868677e-09        9.999987e-01   \n",
       "1517    5.993154e-17  2.025782e-14   2.661710e-13        3.120904e-11   \n",
       "1522    2.355303e-17  6.746580e-13   8.440748e-17        1.377012e-14   \n",
       "1526    1.066409e-15  7.201691e-14   5.221241e-10        5.033192e-11   \n",
       "1528    6.411056e-15  2.580127e-12   8.378771e-15        9.132811e-11   \n",
       "1533    3.300282e-15  2.694800e-18   1.835219e-20        6.126980e-23   \n",
       "1534    2.613066e-11  1.445642e-08   7.926327e-06        1.124311e-09   \n",
       "1535    1.288477e-13  2.326142e-16   1.835840e-14        1.950038e-14   \n",
       "1537    1.620822e-12  2.821214e-07   4.118737e-12        9.999986e-01   \n",
       "1540    6.524499e-12  6.390135e-15   1.482173e-15        9.467809e-16   \n",
       "1542    7.247094e-14  1.376556e-07   2.051661e-13        6.265986e-08   \n",
       "1546    1.138992e-16  1.135046e-11   8.421563e-10        4.079166e-13   \n",
       "1553    2.381800e-09  6.324612e-11   1.394373e-09        1.512711e-06   \n",
       "1558    7.104135e-17  1.000000e+00   1.338219e-16        1.703113e-10   \n",
       "1560    2.315470e-08  1.402375e-12   4.550361e-13        3.020845e-13   \n",
       "1564    1.582777e-11  1.105254e-14   2.483515e-09        4.030555e-14   \n",
       "1565    5.710899e-17  5.059225e-11   3.028270e-11        2.381538e-13   \n",
       "1567    5.500723e-10  2.742796e-06   1.978573e-15        8.122959e-09   \n",
       "1573    7.014879e-13  1.139381e-15   5.437242e-22        4.660121e-19   \n",
       "1576    1.175636e-12  9.311637e-13   1.199286e-14        1.736303e-17   \n",
       "1577    1.560831e-14  1.995448e-07   7.830710e-14        5.373228e-09   \n",
       "1579    2.611521e-17  1.641580e-13   2.154722e-11        2.377172e-14   \n",
       "1580    2.335643e-08  2.371070e-13   4.621850e-09        6.074169e-14   \n",
       "1583    1.991471e-22  1.372112e-19   1.474226e-17        1.597324e-16   \n",
       "\n",
       "      Tilia_Tomentosa  Ulmus_Bergmanniana  Viburnum_Tinus  \\\n",
       "id                                                          \n",
       "4        3.116100e-13        5.175158e-17    3.606400e-16   \n",
       "7        7.358572e-13        7.247904e-16    3.402543e-12   \n",
       "9        3.934525e-12        3.051722e-13    4.054768e-17   \n",
       "12       1.164487e-07        1.359063e-03    1.024508e-08   \n",
       "13       2.947929e-11        3.402397e-07    4.202475e-10   \n",
       "16       9.215131e-08        2.484089e-10    5.835687e-10   \n",
       "19       1.488782e-09        1.030237e-15    4.863621e-15   \n",
       "23       1.288701e-15        1.544091e-17    1.991993e-16   \n",
       "24       7.726076e-12        4.496066e-18    4.528057e-21   \n",
       "28       2.315102e-09        1.549911e-13    3.163423e-20   \n",
       "33       1.007988e-17        3.658791e-10    2.020209e-08   \n",
       "36       2.505662e-13        7.972182e-14    6.035194e-13   \n",
       "39       3.049014e-15        3.376430e-16    1.042771e-14   \n",
       "41       2.686263e-08        1.495722e-14    1.710947e-17   \n",
       "44       7.457406e-17        5.227025e-16    5.447494e-12   \n",
       "46       4.677920e-12        3.981257e-14    7.492120e-13   \n",
       "47       4.957739e-11        1.489013e-14    9.819394e-17   \n",
       "51       2.945607e-14        2.081311e-15    2.070722e-15   \n",
       "52       8.065446e-15        2.454948e-11    3.591680e-14   \n",
       "53       9.897714e-18        1.958945e-14    9.139941e-16   \n",
       "57       1.803881e-20        1.636640e-15    1.266608e-09   \n",
       "59       5.081167e-14        1.657441e-14    1.381596e-07   \n",
       "62       3.613535e-16        3.961770e-13    3.084394e-13   \n",
       "65       1.671860e-11        9.104220e-08    9.503141e-07   \n",
       "68       2.273274e-13        7.045400e-12    8.291578e-18   \n",
       "70       5.281668e-22        4.188845e-18    4.931130e-14   \n",
       "74       6.246726e-17        5.397668e-15    5.972485e-12   \n",
       "77       8.080799e-19        2.196841e-17    2.236530e-19   \n",
       "79       2.791820e-18        6.579154e-17    2.351152e-20   \n",
       "86       1.896076e-21        1.213178e-18    1.182124e-14   \n",
       "...               ...                 ...             ...   \n",
       "1493     1.877295e-14        5.732262e-07    2.652025e-10   \n",
       "1495     4.335020e-14        6.507226e-14    7.962058e-15   \n",
       "1497     2.300294e-09        4.989239e-08    3.793966e-14   \n",
       "1498     1.508352e-10        1.148279e-08    1.213848e-07   \n",
       "1503     4.248125e-11        3.120527e-12    6.098750e-12   \n",
       "1510     5.365640e-12        2.224549e-06    7.796701e-10   \n",
       "1513     4.919524e-13        6.013689e-13    1.109799e-14   \n",
       "1517     1.857066e-18        1.127345e-16    7.404645e-19   \n",
       "1522     1.660151e-15        2.490201e-18    3.056813e-14   \n",
       "1526     2.532269e-14        2.151262e-17    7.692921e-14   \n",
       "1528     7.615978e-18        5.448072e-16    9.248255e-12   \n",
       "1533     5.895878e-12        2.427059e-16    9.567590e-14   \n",
       "1534     3.759798e-09        1.876625e-08    9.228703e-16   \n",
       "1535     1.083543e-11        6.624015e-14    7.345137e-13   \n",
       "1537     3.079923e-12        5.586273e-12    1.288266e-14   \n",
       "1540     4.458600e-15        9.155572e-16    2.815659e-19   \n",
       "1542     1.689670e-13        1.324510e-14    2.418313e-18   \n",
       "1546     1.716784e-16        5.598427e-17    4.307538e-13   \n",
       "1553     1.239112e-09        1.879229e-15    5.880101e-16   \n",
       "1558     2.024396e-18        1.101755e-15    3.428683e-18   \n",
       "1560     1.218311e-08        1.092077e-08    8.900152e-10   \n",
       "1564     8.887296e-13        1.339163e-13    3.329998e-09   \n",
       "1565     4.829297e-18        2.542893e-16    2.363927e-11   \n",
       "1567     1.838869e-19        2.249433e-14    2.059629e-15   \n",
       "1573     3.584236e-17        2.279172e-12    1.620015e-11   \n",
       "1576     1.383317e-13        2.009885e-12    2.494133e-17   \n",
       "1577     1.231009e-07        1.272548e-10    3.576942e-15   \n",
       "1579     2.398065e-18        6.095616e-17    2.039829e-13   \n",
       "1580     3.857699e-11        1.777245e-13    1.733183e-14   \n",
       "1583     2.339374e-16        4.684897e-19    1.200517e-20   \n",
       "\n",
       "      Viburnum_x_Rhytidophylloides  Zelkova_Serrata  \n",
       "id                                                   \n",
       "4                     1.711030e-11     6.657540e-18  \n",
       "7                     1.270131e-16     3.994680e-16  \n",
       "9                     1.650489e-11     8.774527e-09  \n",
       "12                    5.079954e-12     9.817265e-06  \n",
       "13                    8.195366e-12     3.720808e-14  \n",
       "16                    5.496554e-06     3.787366e-09  \n",
       "19                    4.026451e-11     7.848215e-10  \n",
       "23                    8.668355e-15     1.697510e-12  \n",
       "24                    1.178677e-18     7.821341e-19  \n",
       "28                    1.284225e-17     7.948440e-14  \n",
       "33                    2.916461e-08     1.007804e-11  \n",
       "36                    6.344118e-11     1.176658e-10  \n",
       "39                    4.552028e-12     7.101946e-15  \n",
       "41                    1.296577e-14     1.204350e-11  \n",
       "44                    1.261569e-17     2.384301e-11  \n",
       "46                    2.962433e-09     1.783887e-10  \n",
       "47                    8.209071e-18     1.219728e-17  \n",
       "51                    4.766349e-15     5.047050e-13  \n",
       "52                    3.025734e-17     1.108393e-10  \n",
       "53                    9.664176e-14     7.140145e-15  \n",
       "57                    9.859725e-18     1.173085e-16  \n",
       "59                    3.413611e-15     2.701047e-08  \n",
       "62                    5.205927e-18     9.840851e-20  \n",
       "65                    2.706464e-07     6.524240e-15  \n",
       "68                    2.648222e-15     2.151844e-11  \n",
       "70                    1.854321e-17     5.566117e-15  \n",
       "74                    3.108843e-16     5.225053e-13  \n",
       "77                    1.460225e-20     8.232171e-18  \n",
       "79                    4.227329e-11     3.455591e-12  \n",
       "86                    8.128915e-17     4.756759e-15  \n",
       "...                            ...              ...  \n",
       "1493                  4.759859e-09     5.686904e-13  \n",
       "1495                  1.817717e-19     6.565551e-13  \n",
       "1497                  1.538821e-10     2.164320e-13  \n",
       "1498                  4.699070e-14     1.531802e-10  \n",
       "1503                  5.683337e-12     1.367342e-13  \n",
       "1510                  3.186961e-11     1.012459e-13  \n",
       "1513                  2.382722e-14     2.411386e-12  \n",
       "1517                  2.312901e-19     3.494384e-20  \n",
       "1522                  4.666743e-17     3.629625e-11  \n",
       "1526                  9.881657e-14     2.048114e-14  \n",
       "1528                  2.528343e-11     8.635667e-10  \n",
       "1533                  1.550148e-16     1.927212e-15  \n",
       "1534                  1.639717e-16     4.531851e-10  \n",
       "1535                  1.160792e-13     5.068391e-16  \n",
       "1537                  1.170287e-13     6.398837e-14  \n",
       "1540                  1.020880e-14     2.257140e-11  \n",
       "1542                  1.633271e-12     5.152118e-15  \n",
       "1546                  6.100846e-16     9.073183e-10  \n",
       "1553                  5.364567e-15     8.808430e-11  \n",
       "1558                  1.001852e-10     7.753925e-14  \n",
       "1560                  1.464883e-10     8.165475e-11  \n",
       "1564                  3.538699e-16     6.236393e-12  \n",
       "1565                  6.451919e-14     7.762921e-09  \n",
       "1567                  5.725772e-07     2.335336e-14  \n",
       "1573                  2.651923e-10     1.846844e-18  \n",
       "1576                  8.610624e-13     4.773474e-10  \n",
       "1577                  5.204717e-15     4.233081e-09  \n",
       "1579                  2.355133e-15     2.341982e-11  \n",
       "1580                  3.682359e-13     2.326030e-13  \n",
       "1583                  6.421883e-21     2.479388e-16  \n",
       "\n",
       "[594 rows x 99 columns]"
      ]
     },
     "execution_count": 378,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "yPred"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [default]",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}