Reddit_Predicting_Comments.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Using Reddit's API to Predict Number of Comments\n",
    "*Connor Foley*, Data Science Immersive, General Assembly NYC, June 2018"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests\n",
    "import json\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import time\n",
    "from sklearn.model_selection import train_test_split, GridSearchCV, cross_val_score\n",
    "from sklearn.ensemble import RandomForestClassifier, ExtraTreesClassifier\n",
    "from sklearn.pipeline import Pipeline, FeatureUnion\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer, TfidfTransformer\n",
    "from sklearn.naive_bayes import BernoulliNB\n",
    "from sklearn.base import TransformerMixin, BaseEstimator\n",
    "from sklearn.metrics import classification_report, confusion_matrix\n",
    "import matplotlib\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 1: Define the Problem\n",
    "In this project, I am looking at the /hot subreddit of reddit.com, and building a model that accurately predicts whether or not a post will become \"hot,\" having gained a high number of comments.\n",
    "\n",
    "The target we are predicting is the number of comments a post has, which I binarized based on the mean. The required features for this project are subreddit name, post title, and time of post. However, I incorporated and created several other features I thought could be of importance."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 2: Scrape Post Data from Reddit's API"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "url = \"http://www.reddit.com/hot.json\"\n",
    "headers = {'User-agent': 'Con bot 0.1'}\n",
    "res = requests.get(url, headers=headers)\n",
    "res.status_code"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "reddit = res.json()\n",
    "print(reddit.keys())\n",
    "print(reddit['data'].keys())\n",
    "print(reddit['kind'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "## Brief exploration of the JSON\n",
    "sorted(reddit['data']['children'][0]['data'].keys())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here I pull multiple pages of post data, extract the JSON data, and store it in a list of dictionaries so that I can convert it into a DataFrame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(reddit['data']['after'])\n",
    "param = {'after': 't3_8mrd11'}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "posts = []\n",
    "after = None\n",
    "for i in range(168):\n",
    "    if i % 12 == 0:\n",
    "        print('{} of 168'.format(i)) \n",
    "    if after == None:\n",
    "        params = {}\n",
    "    else: \n",
    "        params = {'after': after}\n",
    "    res = requests.get(url, params=params, headers=headers)\n",
    "    if res.status_code == 200:\n",
    "        reddit = res.json()\n",
    "        posts.extend(reddit['data']['children'])\n",
    "        after = reddit['data']['after']\n",
    "    else:\n",
    "        print(res.status_code)\n",
    "        break\n",
    "    time.sleep(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "len(set(p['data']['name'] for p in posts))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I pulled 4200 posts. The length above is 4045, meaning there are 155 duplicates.\n",
    "\n",
    "Below I save everything I have scraped to `posts_df`, and then run a for loop to extract the data I know I will definitely need from the list of posts to a nice, cleaned DataFrame. I saved both to a .csv file as I went."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "posts_df = pd.DataFrame(posts)\n",
    "posts_df.to_csv('./data/posts.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "needed_data = []\n",
    "for n in range(4200):\n",
    "    post_dict = {}\n",
    "    post_data = posts_df.iloc[n,0]\n",
    "    post_dict['title'] = post_data['title']\n",
    "    post_dict['subreddit'] = post_data['subreddit']\n",
    "    post_dict['created_utc'] = post_data['created_utc']\n",
    "    post_dict['num_comments'] = post_data['num_comments']\n",
    "    needed_data.append(post_dict)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "needed = pd.DataFrame(needed_data)\n",
    "needed.to_csv('./data/needed.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 3: Explore the Data\n",
    "#### Load the .csv File\n",
    "Here is where you may start running the cells."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>data</th>\n",
       "      <th>kind</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>{'is_crosspostable': False, 'subreddit_id': 't...</td>\n",
       "      <td>t3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>{'is_crosspostable': False, 'subreddit_id': 't...</td>\n",
       "      <td>t3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>{'is_crosspostable': False, 'subreddit_id': 't...</td>\n",
       "      <td>t3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>{'is_crosspostable': False, 'subreddit_id': 't...</td>\n",
       "      <td>t3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>{'is_crosspostable': False, 'subreddit_id': 't...</td>\n",
       "      <td>t3</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                data kind\n",
       "0  {'is_crosspostable': False, 'subreddit_id': 't...   t3\n",
       "1  {'is_crosspostable': False, 'subreddit_id': 't...   t3\n",
       "2  {'is_crosspostable': False, 'subreddit_id': 't...   t3\n",
       "3  {'is_crosspostable': False, 'subreddit_id': 't...   t3\n",
       "4  {'is_crosspostable': False, 'subreddit_id': 't...   t3"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "posts_df = pd.read_csv('./data/posts.csv') \n",
    "posts_df.drop('Unnamed: 0', axis=1, inplace=True)\n",
    "posts_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Unfortunately, the way I saved the .csv for all the data I scraped resulted in my 'data' column containing the JSON dictionaries inside strings. Thankfully, there's a handy import called ast to fix this. I converted the strings into dictionaries and saved them to a list, in which I used a for loop to extract the extra features I wanted to include and add them to my already cleaned DataFrame of required data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import ast\n",
    "\n",
    "posts = posts_df['data']\n",
    "for n in range(len(posts)):\n",
    "    posts[n] = ast.literal_eval(posts[n])\n",
    "type(posts[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "redd = pd.read_csv('./data/needed.csv') # My cleaned DF\n",
    "redd.drop('Unnamed: 0', axis=1, inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "new_cols = ['num_crossposts', 'is_video', 'subreddit_subscribers', 'score', 'permalink']\n",
    "other_data = []\n",
    "for n in range(4200):\n",
    "    post_dict = {}\n",
    "    post_data = posts[n]\n",
    "    for col in new_cols:\n",
    "        post_dict[col] = post_data[col]\n",
    "    other_data.append(post_dict)\n",
    "    others_df = pd.DataFrame(other_data)\n",
    "    reddit = pd.concat([redd, others_df], axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I included permalink in order to drop duplicate posts. By dropping them by the permalink column, I ensured I would not drop a false duplicate."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(4045, 9)\n"
     ]
    }
   ],
   "source": [
    "reddit.drop_duplicates('permalink', inplace=True)\n",
    "print(reddit.shape)\n",
    "reddit.drop('permalink', axis=1, inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "reddit['created_utc'] = time.time() - reddit['created_utc'] # Relative time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>created_utc</th>\n",
       "      <th>num_comments</th>\n",
       "      <th>subreddit</th>\n",
       "      <th>title</th>\n",
       "      <th>is_video</th>\n",
       "      <th>num_crossposts</th>\n",
       "      <th>score</th>\n",
       "      <th>subreddit_subscribers</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>541880.459872</td>\n",
       "      <td>200</td>\n",
       "      <td>blackpeoplegifs</td>\n",
       "      <td>Destiny calls</td>\n",
       "      <td>False</td>\n",
       "      <td>4</td>\n",
       "      <td>14867</td>\n",
       "      <td>327189</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>544896.459872</td>\n",
       "      <td>3113</td>\n",
       "      <td>MemeEconomy</td>\n",
       "      <td>This format can work for a variety of things. ...</td>\n",
       "      <td>False</td>\n",
       "      <td>7</td>\n",
       "      <td>38179</td>\n",
       "      <td>534642</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>541610.459872</td>\n",
       "      <td>224</td>\n",
       "      <td>StarWars</td>\n",
       "      <td>So I got married last night...</td>\n",
       "      <td>False</td>\n",
       "      <td>1</td>\n",
       "      <td>12216</td>\n",
       "      <td>881134</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>543846.459872</td>\n",
       "      <td>1321</td>\n",
       "      <td>pics</td>\n",
       "      <td>This guy holding an umbrella over a soldier st...</td>\n",
       "      <td>False</td>\n",
       "      <td>6</td>\n",
       "      <td>32034</td>\n",
       "      <td>18677607</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>546286.459872</td>\n",
       "      <td>597</td>\n",
       "      <td>nostalgia</td>\n",
       "      <td>The family computer</td>\n",
       "      <td>False</td>\n",
       "      <td>1</td>\n",
       "      <td>25514</td>\n",
       "      <td>408323</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     created_utc  num_comments        subreddit  \\\n",
       "0  541880.459872           200  blackpeoplegifs   \n",
       "1  544896.459872          3113      MemeEconomy   \n",
       "2  541610.459872           224         StarWars   \n",
       "3  543846.459872          1321             pics   \n",
       "4  546286.459872           597        nostalgia   \n",
       "\n",
       "                                               title  is_video  \\\n",
       "0                                      Destiny calls     False   \n",
       "1  This format can work for a variety of things. ...     False   \n",
       "2                     So I got married last night...     False   \n",
       "3  This guy holding an umbrella over a soldier st...     False   \n",
       "4                                The family computer     False   \n",
       "\n",
       "   num_crossposts  score  subreddit_subscribers  \n",
       "0               4  14867                 327189  \n",
       "1               7  38179                 534642  \n",
       "2               1  12216                 881134  \n",
       "3               6  32034               18677607  \n",
       "4               1  25514                 408323  "
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reddit.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Identify the Target\n",
    "We want to create a binary target from the num_comments column. First, let's look at the distribution. It is incredibly skewed right, with most posts having a small number of comments.\n",
    "\n",
    "The median is only 19, while the maximum value is 10,546 comments. The distribution is so heavily skewed that the 3rd quartile is less than the mean. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/anaconda3/envs/dsi/lib/python3.6/site-packages/matplotlib/axes/_axes.py:6462: UserWarning: The 'normed' kwarg is deprecated, and has been replaced by the 'density' kwarg.\n",
      "  warnings.warn(\"The 'normed' kwarg is deprecated, and has been \"\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAssAAAE8CAYAAAA2d/znAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAHJxJREFUeJzt3XuQ5WV95/H3ufRtZnqagWmuEdGoXzEKuLIKygiWuCiJRuNuFeumspKFkFWLjW5tYqIxmkUtk7hWscZN4par2RJrvRQJEiG63jIOsESCCF6+phFFQGBmGGZ6pqd7+nL2j3N6OMz0M31m+jQ9feb9qurqc37P8+vz/M635/Tn98xzzq/SaDSQJEmSdLDqSg9AkiRJOloZliVJkqQCw7IkSZJUYFiWJEmSCgzLkiRJUoFhWZIkSSqor/QADmXr1vEV/Vy7DRvWsGPHxEoOQV1iLXuHtewd1rJ3WMvecSzXcnR0uLLQdmeWD6Fer630ENQl1rJ3WMveYS17h7XsHdbyYIZlSZIkqcCwLEmSJBUYliVJkqQCw7IkSZJUYFiWJEmSCgzLkiRJUoFhWZIkSSowLEuSJEkFhmVJkiSpwLAsSZIkFRiWJUmSpIL6Sg/gaHbzrT9hfPdkx/0vOue05RuMJEmSnnLOLEuSJEkFhmVJkiSpwLAsSZIkFRiWJUmSpALDsiRJklRgWJYkSZIKDMuSJElSgWFZkiRJKjAsS5IkSQWGZUmSJKnAsCxJkiQVGJYlSZKkAsOyJEmSVGBYliRJkgoMy5IkSVKBYVmSJEkqMCxLkiRJBYZlSZIkqcCwLEmSJBXUF+sQEVXgY8DZwBRwRWaOtbVfCVwFzADXZOaNEbERuA4YAh4CLs/Mibaf93fA32bmX3T5eCRJkqSu6WRm+fXAYGaeD7wT+PB8Q0ScDFwNvAy4BPhgRAwA7wGuy8xNwJ00w/S8a4DjuzN8SZIkafl0EpYvAG4GyMzbgHPb2l4MbMnMqczcCYwBZ7XvA9wEXAwQEf8amGttkyRJko5qnYTl9cDOtvuzEVEvtI0DIwdsHwdGIuL5wJtozjpLkiRJR71F1ywDu4DhtvvVzJwptA0Dj7dt39u27TeA04CvAWcA+yLiJ5l5MwUbNqyhXq91diTLYWw7w+sGO+4+Ojq8eCetGOvTO6xl77CWvcNa9g5r+WSdhOUtwGuBz0bEecDdbW23A++PiEFgADgTuKe1z6XAJ4HXAJsz80PzO0XEe4GHDxWUAXbsmOj4QJbL+O7Jjvtu3Tq+jCPRUoyODlufHmEte4e17B3Wsnccy7UsnSR0EpavB14VEbcAFeDyiHgHMJaZN0TEtcBmmks63pWZkxFxDfCp1idlbKO5/EKSJElaVSqNRmOlx1C0dev4ig7ujrHthzWzfNE5py3jaLQUx/KZcq+xlr3DWvYOa9k7juVajo4OVxba7kVJJEmSpALDsiRJklRgWJYkSZIKDMuSJElSgWFZkiRJKjAsS5IkSQWGZUmSJKnAsCxJkiQVGJYlSZKkAsOyJEmSVGBYliRJkgoMy5IkSVKBYVmSJEkqMCxLkiRJBYZlSZIkqcCwLEmSJBUYliVJkqQCw7IkSZJUYFiWJEmSCgzLkiRJUoFhWZIkSSowLEuSJEkFhmVJkiSpwLAsSZIkFRiWJUmSpALDsiRJklRgWJYkSZIKDMuSJElSgWFZkiRJKjAsS5IkSQWGZUmSJKnAsCxJkiQVGJYlSZKkAsOyJEmSVGBYliRJkgoMy5IkSVKBYVmSJEkqMCxLkiRJBYZlSZIkqcCwLEmSJBUYliVJkqQCw7IkSZJUYFiWJEmSCgzLkiRJUoFhWZIkSSowLEuSJEkFhmVJkiSpwLAsSZIkFRiWJUmSpALDsiRJklRgWJYkSZIK6ot1iIgq8DHgbGAKuCIzx9rarwSuAmaAazLzxojYCFwHDAEPAZdn5kREvBV4M9AA/jgzb+zy8UiSJEld08nM8uuBwcw8H3gn8OH5hog4GbgaeBlwCfDBiBgA3gNcl5mbgDuBq1oB+i3AS4FXAv8jIirdPBhJkiSpmzoJyxcANwNk5m3AuW1tLwa2ZOZUZu4ExoCz2vcBbgIuzsxtwNmZOQ2cDDyemY3uHIYkSZLUfZ2E5fXAzrb7sxFRL7SNAyMHbJ/fRmbORMTbgNuAzy9h3JIkSdKyW3TNMrALGG67X83MmULbMPB42/a9bdsAyMyPRsRfATdFxCsy8+ulB96wYQ31eq2jA1kWY9sZXjfYcffR0eHFO2nFWJ/eYS17h7XsHdayd1jLJ+skLG8BXgt8NiLOA+5ua7sdeH9EDAIDwJnAPa19LgU+CbwG2BwRAXwQeCMwTfPNgnOHeuAdOyYO51iWxfjuyY77bt06vowj0VKMjg5bnx5hLXuHtewd1rJ3HMu1LJ0kdLIM43pgMiJuAT4CvD0i3hERr8vMh4Frgc3A14B3ZeYkcA1wWURsAc4HPpqZCdwF3ArcAtyWmd9c4nFJkiRJy6bSaBy977HbunV8RQd3x9j2w5pZvuic05ZxNFqKY/lMuddYy95hLXuHtewdx3ItR0eHF/yUNi9KIkmSJBUYliVJkqQCw7IkSZJUYFiWJEmSCgzLkiRJUoFhWZIkSSowLEuSJEkFhmVJkiSpwLAsSZIkFRiWJUmSpALDsiRJklRgWJYkSZIKDMuSJElSgWFZkiRJKjAsS5IkSQWGZUmSJKnAsCxJkiQVGJYlSZKkAsOyJEmSVGBYliRJkgoMy5IkSVKBYVmSJEkqMCxLkiRJBYZlSZIkqcCwLEmSJBUYliVJkqQCw7IkSZJUYFiWJEmSCgzLkiRJUoFhWZIkSSowLEuSJEkFhmVJkiSpwLAsSZIkFRiWJUmSpALDsiRJklRgWJYkSZIKDMuSJElSgWFZkiRJKjAsS5IkSQWGZUmSJKnAsCxJkiQVGJYlSZKkAsOyJEmSVGBYliRJkgoMy5IkSVKBYVmSJEkqMCxLkiRJBYZlSZIkqcCwLEmSJBUYliVJkqQCw7IkSZJUUF+sQ0RUgY8BZwNTwBWZOdbWfiVwFTADXJOZN0bERuA6YAh4CLg8Myci4u3AZa1dv5SZ7+vq0UiSJEld1MnM8uuBwcw8H3gn8OH5hog4GbgaeBlwCfDBiBgA3gNcl5mbgDuBqyLimcC/A14KnA/8q4g4q5sHI0mSJHVTJ2H5AuBmgMy8DTi3re3FwJbMnMrMncAYcFb7PsBNwMXAz4BXZ+ZsZs4BfcBkV45CkiRJWgaLLsMA1gM72+7PRkQ9M2cWaBsHRg7YPg6MZOY0sC0iKsCfAndm5o8O9cAbNqyhXq91diTLYWw7w+sGO+4+Ojq8jIPRUlmf3mEte4e17B3WsndYyyfrJCzvAtqftWorKC/UNgw83rZ9b9s2ImIQ+ATNAP2WxR54x46JDoa3vMZ3dz75vXXr+DKOREsxOjpsfXqEtewd1rJ3WMvecSzXsnSS0MkyjC3ApQARcR5wd1vb7cCmiBiMiBHgTOCe9n2A1wCbWzPKfwvclZlXZebskRyIJEmS9FTpZGb5euBVEXELUAEuj4h3AGOZeUNEXAtsphm835WZkxFxDfCp1idlbAPeRPONghcCAxHxmtbP/v3MvLXLxyRJkiR1xaJhufVmvN8+YPMP29o/Dnz8gH0eAV59wD7XA50vAJYkSZJWmBclkSRJkgoMy5IkSVKBYVmSJEkqMCxLkiRJBYZlSZIkqcCwLEmSJBUYliVJkqQCw7IkSZJUYFiWJEmSCgzLkiRJUoFhWZIkSSowLEuSJEkFhmVJkiSpwLAsSZIkFRiWJUmSpALDsiRJklRgWJYkSZIKDMuSJElSgWFZkiRJKjAsS5IkSQWGZUmSJKnAsCxJkiQVGJYlSZKkAsOyJEmSVGBYliRJkgoMy5IkSVKBYVmSJEkqMCxLkiRJBYZlSZIkqcCwLEmSJBUYliVJkqQCw7IkSZJUYFiWJEmSCgzLkiRJUoFhWZIkSSowLEuSJEkFhmVJkiSpwLAsSZIkFRiWJUmSpALDsiRJklRgWJYkSZIKDMuSJElSgWFZkiRJKjAsS5IkSQWGZUmSJKnAsCxJkiQVGJYlSZKkAsOyJEmSVGBYliRJkgoMy5IkSVKBYVmSJEkqqC/WISKqwMeAs4Ep4IrMHGtrvxK4CpgBrsnMGyNiI3AdMAQ8BFyemROt/qPALcALMnOyy8cjSZIkdU0nM8uvBwYz83zgncCH5xsi4mTgauBlwCXAByNiAHgPcF1mbgLupBmmiYhLgC8DJ3XzICRJkqTl0ElYvgC4GSAzbwPObWt7MbAlM6cycycwBpzVvg9wE3Bx6/Zc6/ZjSx+6JEmStLwWXYYBrAd2tt2fjYh6Zs4s0DYOjBywfX4bmfkVgIjoaHAbNqyhXq911HdZjG1neN1gx91HR4eXcTBaKuvTO6xl77CWvcNa9g5r+WSdhOVdQPuzVm0F5YXahoHH27bvbdt22HbsmDiS3bpqfHfny6q3bh1fxpFoKUZHh61Pj7CWvcNa9g5r2TuO5VqWThI6WYaxBbgUICLOA+5ua7sd2BQRgxExApwJ3NO+D/AaYPORDVuSJElaOZ3MLF8PvCoibgEqwOUR8Q5gLDNviIhraYbhKvCuzJyMiGuAT7U+KWMb8KZlGr8kSZK0bCqNRmOlx1C0dev4ig7ujrHth7UM46JzTlvG0WgpjuX/Vuo11rJ3WMveYS17x7Fcy9HR4cpC270oiSRJklRgWJYkSZIKDMuSJElSgWFZkiRJKjAsS5IkSQWGZUmSJKnAsCxJkiQVGJYlSZKkAsOyJEmSVGBYliRJkgoMy5IkSVKBYVmSJEkqMCxLkiRJBYZlSZIkqcCwLEmSJBUYliVJkqQCw7IkSZJUYFiWJEmSCgzLkiRJUoFhWZIkSSowLEuSJEkFhmVJkiSpwLAsSZIkFRiWJUmSpALDsiRJklRgWJYkSZIKDMuSJElSgWFZkiRJKjAsS5IkSQWGZUmSJKnAsCxJkiQVGJYlSZKkAsOyJEmSVGBYliRJkgoMy5IkSVKBYblLHh+f4uNf/D5fveOBlR6KJEmSuqS+0gNY7fbsneause3c++BOGsDtP3iE552xgVNOWLvSQ5MkSdISObN8hPZNz3JHbuVvNt/H2IM7GVnXz6XnPZ3ZuQaf/drYSg9PkiRJXeDM8hH65nce4ufbJ1gzWOeFz97IM05dzyvOOY0fP7STu+7dzj33bef5zzhhpYcpSZKkJXBm+QjsnZrh59sn2DgyyBs2PYNfPG2EaqVCpVLhslc+mwrwf746xuzc3EoPVZIkSUtgWD4CP3tkNwBnnDJMrfbkp/D0k4bZdPYpPLhtD//wnYdWYniSJEnqEsPyEbj/0XEATj9xeMH2N7z8Fxnsr3H95vuYmJx+KocmSZKkLjIsH6Z907M8vH2C49cPsG5N34J9Rtb288vnP53de6e5YctPntoBSpIkqWt8g99hemDrHuYazeUWB/rGdx7cf3uwv8a6oT6+8u2fMThQ47h1Awf1v+ic05Z1rJIkSVoaZ5YP088emV+Cse6Q/Wq1Ki+KURoN+NKtP+WusW1Mz/iGP0mSpNXEsHwYZmbneHDbHobX9DGyrn/R/qeftI7zn38S9VqVu8a2Nz+T+YGdNBqNp2C0kiRJWiqXYRyGn2+fYGa2weknDVOpVBbtX6lUePYvHMcZJ6/nnh9v5/s/2cEt9zzMD366g01nnfIUjFiSJElL4czyYbh/fgnGSYdegnGgvnqVFz5nlNdvegbPPHU9O8anuOn/3c89P96+HMOUJElSlxiWOzQ31+CBR/cwNFBj48jgEf2MtUN9XHDWKWw6+xRm5xp85HN38dU7HujySCVJktQtLsPo0KM79jI1PctznnZcR0swDuUZp6xn3VAfW777cz79lR/x8GMTXPbKZ1GtVHh0x15+/PNd3PfQLvbum2HjyBAbRwbZODLICSODHL9+kOoSH1+SJEmdWTQsR0QV+BhwNjAFXJGZY23tVwJXATPANZl5Y0RsBK4DhoCHgMszc2Khvt0+oOVypEswSkaPG+Lic5/G1/7pAb56xwN8+4ePsnffDPumD/2JGRtHBnnFC0/jgrNOYXjN4m8ylCRJ0pHrZGb59cBgZp4fEecBHwZ+FSAiTgauBs4FBoFvRcRXgPcA12XmJyPincBVEfGZhfpm5lTXj2oJZmbnuCO3ctrGtfs/taLRaHD/o7vpr1c5+fg1XXusdWv6ePV5p7P5rp/z4NY9rBvq49SNaxltzSYP9NfYvXeaPXun2b13moH+Ot8d28bnvnEv12++j3/53BO56IWnMrK2n71Ts0xMzTA5NcPE1Ax7p2bYu2+2+X1qhtm5BiNr+1m/tp+R1tfaoT766lX6alXqre+D/bUlz5xLkiT1ik7C8gXAzQCZeVtEnNvW9mJgSyvwTkXEGHBWa58PtPrc1Lp9b6HvP3blSLok73+cv7zhewCsHaxz8glrGFnbz8TkDM88dT3VaneDZH+9xitf9AvMzM5Rrx28hHz92ifPHj/rtPXc++Au8v4d3Pq9h7n1ew93eTxVjl8/yAnrBzhhZJANw4PUaxUajeZJQwOgAXP7TySgQYP5T8Oba26gAdSqFQb7awz21xnoqzHQX6PRaDA9M8f07BzTM3PMzjYYGqixbqif4TV9rBvqY2igzuS+GSb3NU8A9k7OsG9mltm5BnOtr5m5BvValaGBGmsG6gy1vvpqVWrVCtVqhVq1QqVaYXa2wVytxqOPTTAz23zMSgWqlQqVSvNTS9rvN78/0VZtfacC09NzTE7P7h/f9PQcfX3Nk4zBvuax9vdVj+iEY67RYM/eaXbt2cfOPfvYuXsfE1MzrBms7z/BWb+2n6GBOlPTs0ztm2Wy9dVoNPY/14P9Nfr7midajzw2wSM7Jnj4sQm275xk3Zp+Tj5+iJM2rOGk49dw/PAAk/tmn3RSNj07x9rBZi3WDfWxdqhOrVrdfzK2d98Mk1OzVKuV1vNeY6i/zkB/jT2TM+zaPdUc/5597J2aYd1QHyPrBvYfw9BAnZnZOWbmGsy2fg8qlQr1WoVarUq9WqFeq7JvZpaJyebJ3sRk8/nu76tyyuOTTE9OMzRYZ7Cv+Ts114DZuTnmWr+nABWAVh0q8/dbdyocXN9KBeYa7P8dm2u0vvbfb7YB1GuV/SeYtVqVA18WDvpwyMZi7Y1Dtx/cZYH2Q3dYqLmxyMAO3GWxcc9vmv+3Ott6Hqut+vbVq9RbJ+cAfePN35VG44nXkPnbDZ54LWnMH1/jidutu9B6XWq01f5Jr0fzbfOvUwe8Zh34s+Z4os/849Vr1eZrWF/ze39/zeVwBxjYs4/de6dXehjqgvZazjUazMzMsW9mjn3Ts82/23MN+upV+utV+vpq9NerNBrsn5ybn6yrVirNvw8DddYM1Pf/Xdq+a5IduyZ5bHyK8Ylp1q/t5/j1Axw/3Mweo8cN0d9XW+Fn4ck6CcvrgZ1t92cjop6ZMwu0jQMjB2xfaFv79qPKmWds4KrX/RJ33buN7/zzNu59cNf+tqctciGSpVgoKC+kv6/GmWds4LlPP46HH5vgvofGadCgv15rzhIf8As8v61aqewPOfO/zPtav/Szs63vcw32Ts2wY3yKhx+bWLZjlSRJWsjwmj7+7C0vpa9+9ATmTsLyLqD92s7VVlBeqG0YeLxt+94Fth3Yt2h0dHhFTt1/5cT1/MqFz1qJh5YkSdJRpJPpzC3ApQCtNct3t7XdDmyKiMGIGAHOBO5p3wd4DbD5EH0lSZKko1JlsXVubZ+GcRbNZX+X0wzCY5l5Q+sTLn6LZvD+QGZ+ISJOAj5Fc/Z4G/CmzNyzUN9lOi5JkiRpyRYNy5IkSdKxyiv4SZIkSQWGZUmSJKnAsCxJkiQVdPLRcceUxS7vraNHRPQBnwDOAAaAa4DvA5+keS2Be4C3ZuZcRPwR8Ms0L7X+O5l5e0Q8a6G+T/FhqE1EnAjcAbyKZq0+ibVcdSLi94HXAf00X0+/ibVcdVqvsZ+i+Ro7C1yJ/y5XnYh4CfChzLyoVJPDqd9CfZ/yg3qKObN8sP2X9wbeSfPy3jo6/TqwPTM30fyIwo8C/w14d2tbBfjViPgXwIXAS4DLgD9v7X9Q36d4/GrT+sP8lzQ/nx2s5aoUERcBLwVeRrNWT8NarlaXAvXMfCnwx8D7sZarSkT8LvA/gcHWpiXV7xB9e5ph+WBPurw3cO6hu2sFfQ74w7b7M8CLaM5iQfNS6xfTrOmXM7ORmfcD9YgYLfTVyvkz4C+Ah1r3reXqdAnNz+O/HvgicCPWcrX6Ec26VGlehXcaa7na3Av8Wtv9pdav1LenGZYPtuDlvVdqMCrLzN2ZOR4Rw8DngXcDlcyc/zzExS61vlBfrYCIeDOwNTP/vm2ztVydNtKcZPg3wG8Dn6Z55VdrufrsprkE44fAx4Fr8d/lqtK6nsV026al1q/Ut6cZlg92qMt76ygTEU8Dvg7878y8DmhfD7fYpdYX6quV8ZvAqyLiG8A5wF8DJ7a1W8vVYzvw95m5LzMTmOTJf0yt5erxdpq1fA7N9/F8iuY69HnWcvVZ6t/IUt+eZlg+2KEu762jSOtKkV8Gfi8zP9HafGdrzSQ8can1LcAlEVGNiNNpngBtK/TVCsjMl2fmhZl5EfAd4DeAm6zlqvQt4NURUYmIU4G1wFet5aq0gydmER8D+vA1drVbav1KfXuaywsOdj3NGa5beOLy3jo6/QGwAfjDiJhfu/yfgGsjoh/4AfD5zJyNiM3ArTRPEN/a6vufgY+3931KR6/FHFQfa3n0y8wbI+LlwO08UaP7sJar0UeAT7Tq1E/zNffbWMvVbEmvq4fo29O83LUkSZJU4DIMSZIkqcCwLEmSJBUYliVJkqQCw7IkSZJUYFiWJEmSCgzLkiRJUoFhWZLUFRExEhHXr/Q4JKmbDMuSpG7ZALxwpQchSd3kFfwk6Qi0LgP7B8AEcCZwd+v+lzPzjFaf9wJk5nsj4mHgb4CXAA8DnwCuBn4BeHNmfvMQj/V04H8BJ7Ye74rM/G5EXE7zKlsN4A7gbZm5u5PHiohvAP8EXAAMAr9H8wqYzwM+kpkfiYh1wJ8DzwdqwIcy8zMR8Wbg1cDxwDNbx/wW4Frg1Nbs8r8HPgOc3DqM92XmDYf3LEvSynNmWZKO3EuBt9EMy6cDlxyi70nATZn5Qprh9A2ZuQl4L/A7izzOx4AvZObzW/3fHREvAN4FXJiZLwD2AH90mI9VycwXA18A/jvwa8Am4D2t9ncDd2Tmi4CXA++KiGe2HfsbgbOA17bGczXwUGa+AXgD8JPWvv+h9XMladVxZlmSjtw9mfkAQET8gOZM66Hc1Pr+U+Bbbbc3LLLfhcC/BcjMLwFfioi3AV/MzO2tPn9Fc/b5cB6rvc9tmTkB/DQijmttvxhYExG/2bq/Fvil1u1bMnMcICJ+TPPYx9t+9i3AByLiNODvgP+6yDFK0lHJmWVJOnKTbbcbre+Vtm197Z0zc1/b3ZnDeJzp+RsRUYmI53Hw63eFtgmQDh9rsT414Ncz85zMPAc4D7i51XbgsbcfN5n5z8BzgU/TnFW+PSL8myNp1fGFS5K653Hg+IgYjYgBmut6u+EfgMtaty+mOYv8DeB1ETE/m30l8PUuPd68rwH/ESAiTgG+S3O5SckMrcDemvl+X2Z+DngLzfXW67s8PkladoZlSeqencCfAP8I/F/g9i793LcBb4yI7wDvA34rM78LfBD4ZkT8EDiO5hrjbnofMBQR99AMzr+bmfceov8jwP0R8XXgr4GIiLuBzcB/yczHuzw+SVp2lUajsXgvSZIk6RjkG/wk6SgQEX8KvGqBpm9n5hVP9XgkSU3OLEuSJEkFrlmWJEmSCgzLkiRJUoFhWZIkSSowLEuSJEkFhmVJkiSp4P8DEGKYVs9V01cAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 864x360 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fix, ax = plt.subplots(figsize=(12,5))\n",
    "sns.distplot(reddit['num_comments']);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count     4045.000000\n",
       "mean        79.350309\n",
       "std        323.166913\n",
       "min          0.000000\n",
       "25%          7.000000\n",
       "50%         19.000000\n",
       "75%         50.000000\n",
       "max      10546.000000\n",
       "Name: num_comments, dtype: float64"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reddit['num_comments'].describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Top ten largest values: [3147, 3301, 3446, 3627, 3674, 3989, 4534, 5866, 6715, 10546]\n"
     ]
    }
   ],
   "source": [
    "# There are some clear outliers:\n",
    "print(\"Top ten largest values:\", sorted(reddit['num_comments'])[-10:])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Baseline accuracy: 0.8286773794808405\n"
     ]
    }
   ],
   "source": [
    "# Create binary high comment target:\n",
    "reddit['high_comment_mean'] = (reddit.num_comments > 79.35)*1\n",
    "print(\"Baseline accuracy:\", reddit.high_comment_mean.value_counts()[0]/len(reddit.high_comment_mean))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Some Feature Engineering\n",
    "I approached this with the assumption that many of the posts with the most amount of comments would come from a few of the same very popular subreddits, such as /r/pics. Therefore, I decided to use `get_dummies` on the subreddit column. \n",
    "\n",
    "To practice some additional feature engineering, I created two new columns out of the 'title' column: \n",
    "- A column in which the word \"best\" appears in the post title\n",
    "- A column in which the first word is \"when\" (Considering one-liner memes such as _when the bass drops_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "best = []\n",
    "for n in reddit.title:\n",
    "    if \"best\" in n:\n",
    "        best.append(1)\n",
    "    elif \"Best\" in n:\n",
    "        best.append(1)\n",
    "    elif \"BEST\" in n:\n",
    "        best.append(1)\n",
    "    else:\n",
    "        best.append(0)\n",
    "reddit['best']=best"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "when = []\n",
    "for n in reddit.title:\n",
    "    if n[0:4]==\"when\":\n",
    "        when.append(1)\n",
    "    elif n[0:4]==\"When\":\n",
    "        when.append(1)\n",
    "    elif n[0:4]==\"WHEN\":\n",
    "        when.append(1)\n",
    "    else:\n",
    "        when.append(0)\n",
    "reddit['when'] = when"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I created a DataFrame of strictly numerical data and removed the target."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "uint8      1987\n",
       "int64         6\n",
       "float64       1\n",
       "dtype: int64"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "subreddit_dummies = pd.get_dummies(reddit['subreddit'])\n",
    "new_reddit = pd.concat([reddit, subreddit_dummies], axis=1)\n",
    "new_reddit = new_reddit.drop(['subreddit','title','num_comments', 'high_comment_mean'], axis=1)\n",
    "new_reddit['is_video'] = (new_reddit['is_video']==True).astype(int)\n",
    "new_reddit.dtypes.value_counts() "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Step 4: Model\n",
    "With my DataFrame of strictly numerical data, i.e. all features except \"title\", I wanted to see how a Logistic Regression model would score.\n",
    "\n",
    "#### Logistic Regression on Numerical Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Train Test Split\n",
    "X_train, X_test, y_train, y_test = train_test_split(new_reddit, reddit['high_comment_mean'], random_state=24, stratify=reddit['high_comment_mean'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "ss = StandardScaler()\n",
    "logreg = LogisticRegression(random_state=24)\n",
    "pipe = Pipeline([('ss', ss),('logreg', logreg)])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.8784584980237155"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pipe.fit(X_train, y_train)\n",
    "pipe.score(X_test, y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.852820985248438"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# An improvement from the baseline\n",
    "cross_val_score(pipe, X_test, y_test, cv=6).mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.8824110671936759"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# GridSearch to find best parameters:\n",
    "params = {\n",
    "    'logreg__penalty': ['l1','l2'],\n",
    "    'logreg__C': [.3,.4,.5,.6,.7,.8,.9,1.0]\n",
    "}\n",
    "gs = GridSearchCV(pipe, param_grid=params, scoring='accuracy')\n",
    "gs.fit(X_train, y_train)\n",
    "gs.score(X_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is not worth including all the code, but trying a kNN Classifier model on the same features resulted in a worse score than the baseline on the test data, exhibiting a high level of overfitting on the training data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Next Steps\n",
    "I wanted to use some Natural Language Processing tools on the \"title\" column. However, creating a model using CountVectorizer and Bernoulli Naïve Bayes on \"title\" resulted in an accuracy score about the same as the baseline. Therefore, I wanted to try to create a model that combines my numerical data, which I used with the Logistic Regression model, with some Natural Language Processing.\n",
    "\n",
    "The natural conclusion was to use FeatureUnion, nesting two pipelines within a pipeline. Using GridSearchCV, I reasoned that the combination of these models would give me an accuracy score significantly better than the baseline."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### MetaPipeline with FeatureUnion\n",
    "Given that we have unbalanced classes in the target, I wanted to first try a model that will use bootstrapping: Random Forest Classifier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = pd.concat([new_reddit, reddit['title']], axis=1)\n",
    "y = reddit['high_comment_mean']\n",
    "cont_cols = [col for col in X.columns if col !='title']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test, = train_test_split(X, y, random_state=24)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define a class to tell the pipeline which features to take\n",
    "class DfExtract(BaseEstimator, TransformerMixin):\n",
    "    def __init__(self, column):\n",
    "        self.column = column\n",
    "    def fit(self, X, y=None):\n",
    "        return self\n",
    "    def transform(self, X, y=None):\n",
    "        if len(self.column) > 1:\n",
    "            return pd.DataFrame(X[self.column])\n",
    "        else:\n",
    "            return pd.Series(X[self.column[0]]) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [],
   "source": [
    "rf_pipeline = Pipeline([\n",
    "    ('features', FeatureUnion([\n",
    "        ('post_title', Pipeline([\n",
    "            ('text', DfExtract(['title'])),\n",
    "            ('cvec', CountVectorizer(binary=True, stop_words='english')),\n",
    "            ])),\n",
    "        ('num_features', Pipeline([\n",
    "            ('num_cols', DfExtract(cont_cols)),\n",
    "            ('ss', StandardScaler())\n",
    "        ])),\n",
    "    ])),\n",
    "    ('rf', RandomForestClassifier(random_state=24))\n",
    "])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [],
   "source": [
    "parameters = {\n",
    "    'rf__n_estimators': [10,30,40,50,60],\n",
    "    'rf__max_depth': [None,4,5],\n",
    "    'rf__max_features': ['auto', 'log2', 15, 20]\n",
    "}\n",
    "rf_gs = GridSearchCV(rf_pipeline, param_grid=parameters, scoring='accuracy')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.8962450592885376"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rf_gs.fit(X_train, y_train)\n",
    "rf_gs.score(X_test, y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'rf__max_depth': None, 'rf__max_features': 'auto', 'rf__n_estimators': 40}"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rf_gs.best_params_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### However...\n",
    "Logistic Regression ended up giving me a slightly better score, despite the intuition that a bootstrapping model would make more sense for this kind of data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "pipeline = Pipeline([\n",
    "    ('features', FeatureUnion([\n",
    "        ('post_title', Pipeline([\n",
    "            ('text', DfExtract(['title'])),\n",
    "            ('cvec', CountVectorizer(binary=True, stop_words='english')),\n",
    "            ])),\n",
    "        ('num_features', Pipeline([\n",
    "            ('num_cols', DfExtract(cont_cols)),\n",
    "            ('ss', StandardScaler())\n",
    "        ])),\n",
    "    ])),\n",
    "    ('logreg', LogisticRegression(random_state=24))\n",
    "])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "parameters = {\n",
    "    'logreg__penalty': ['l1','l2'],\n",
    "    'logreg__C': [.3,.4,.5,.6,.7,.8,.9,1.0],\n",
    "}\n",
    "gs = GridSearchCV(pipeline, param_grid=parameters, scoring='accuracy')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9051383399209486"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gs.fit(X_train, y_train)\n",
    "gs.score(X_test, y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'logreg__C': 0.8, 'logreg__penalty': 'l1'}"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gs.best_params_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Step 5: Evaluate Model\n",
    "The total scores from the classification report for both models are 0.88-0.91, which is about the same as the accuracy score. Also, given that the optimal penalty for the LogisticRegression model was Lasso, we can conclude that there were irrelevant features in the model.\n",
    "\n",
    "Given that there were unbalanced classes in the target variable, I wanted to include a model that would bootstrap data from an underrepresented class, hence trying RandomForest and LogisticRegression. Both gave around the same score, with LogisticRegression performing slightly better. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 148,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "               precision    recall  f1-score   support\n",
      "\n",
      " Low Comments       0.90      0.99      0.94       845\n",
      "High Comments       0.87      0.44      0.58       167\n",
      "\n",
      "  avg / total       0.89      0.90      0.88      1012\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Classification Report for Random Forest:\n",
    "rf_predictions = rf_gs.predict(X_test)\n",
    "print(classification_report(y_test, rf_predictions, target_names=['Low Comments', 'High Comments']))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 149,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "               precision    recall  f1-score   support\n",
      "\n",
      " Low Comments       0.93      0.96      0.94       845\n",
      "High Comments       0.76      0.62      0.68       167\n",
      "\n",
      "  avg / total       0.90      0.91      0.90      1012\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Classification Report for Logistic Regression:\n",
    "predictions = gs.predict(X_test)\n",
    "print(classification_report(y_test, predictions, target_names=['Low Comments', 'High Comments']))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I used a function from scikitlearn's documentation on confusion matrices to plot one:\n",
    "_This is for the Logistic Regression model._"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Confusion matrix, without normalization\n",
      "[[812  33]\n",
      " [ 63 104]]\n",
      "Normalized confusion matrix\n",
      "[[0.96 0.04]\n",
      " [0.38 0.62]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjEAAAIjCAYAAADycUpkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xm85nPdx/HXOWfMxoyxhyxl+VTIGtknlbWS5KakKIQhusuaCkm6SXdCQpZ2IbcomVv2nSwRPrJkTTHMYgxjZs79x+93uJp7zjKc61zX9bteT4/zONf1u67rd33PcWbOZ97fz+/77eju7kaSJKnVdDZ6AJIkSW+GRYwkSWpJFjGSJKklWcRIkqSWZBEjSZJakkWMJElqScMaPQBJkjT4Rq29/5CtoTLjrlM6huq9apnESJKklmQSI0lSFXVUP6eo/lcoSZIqySRGkqQq6mhIm8qQsoiRJEl1ExELAOcBKwKzgb2AWcC5QDdwHzAhM+dExDeB7crHD8rM2/o6t9NJkiSpnrYFhmXmRsAxwLeBk4AjM3NToAPYPiLWATYHNgB2AU7t78QWMZIkVVFH59B99O0hYFhEdAJjgdeAdYFry8cvBz4EbAJMzMzuzHyifM0SfZ3Y6SRJklRPL1FMJT0ILA58BNgsM3vWsZkGLExR4EyqeV3P8ed6O7FJjCRJVdTRMXQfffsycEVmrgqsSdEfM7zm8THAZGBqeXvu472yiJEkSfX0IjClvP0CsABwV0SML49tA1wP3AhsFRGdEbE80JmZz/d1YqeTJEmqouZZ7O77wNkRcT1FAnMEcAdwZkQMBx4ALszM2eVzbqYIWSb0d+KO7u4h21pBkiQNkVHrf3Xo9k667cSGLEpjEiNJUhW1wWJ3TZM1SZIkzQ+TGEmSqqh5emLqpvpfoSRJqiSTGEmSqsieGEmSpOZkEiNJUhXZEyNJktScTGIkSaoie2IkSZKak0mMJElVZE+MJElSczKJkSSpiuyJkSRJak4mMZIkVZE9MZIkSc3JJEaSpCoyiZEkSWpOJjGSJFVRp1cnSZIkNSWTGEmSqsieGEmSpOZkEiNJUhW5Yq8kSVJzsoiRJEktyekkSZKqyMZeSZKk5mQSI0lSFdnYK0mS1JxMYiRJqiJ7YiRJkpqTSYwkSVVkT4wkSVJzMomRJKmK7ImRJElqTiYxkiRVkT0xkiRJzckkRpKkKrInRpIkqTmZxEiSVEX2xEiSJDUnkxhJkqrInhhJkqTmZBIjSVIVmcRIkiQ1J5MYSZKqyKuTJEmSmpNJjCRJVdQGPTEWMVIFRUQXcCDwaYo/58OBS4FvZOarb+GcvwXeDZycmafM5+vXAw7LzE++mfcfbBGxMHBxZm7Ry+N3A+Mzc/LQjkzSQFnESNX0I2AR4IOZOSUiFgR+AZwF7PYmz7kssBWwYGbOnt8XZ+YdQFMUMKVFgPV7ezAz1xrCsUiDrw16YixipIqJiBWBXYGlM3MqQGZOj4h9gI3L5ywMnAqsBXQDlwNHZOasiHgFOB7YElga+C/g58AfgQWAP0fEjsDDwBKZ+Xx5zm5gCeAV4BxgFWAO8Gfgi8BmwCmZufr8vn9m/mgeX+crwEnAh4CFgKOAnYA1gGeAj5Zf9+fL9x8OLAocX57vHGBUmbisC7wMXAKsWX7/bi+/ngkUxdum5f07gV0z8+r5+h8jadBVf8JMaj/rAn/tKWB6ZOazmXlRefdkYBLFL/z1KH5xf7V8bATwfGZuRJGcfB94DdgWmJGZa2XmI328/w7AmDLJeF957J1zPWe+3j8iRs7jfUYAz2bm+sB5FCnTQcB7gIWB7SNiIWAvYNvMXBvYmaIoA9ij5uuZTTnllplRpkY9ji2//oOBn1EUYhYwan4dnUP30SAWMVL1zKH/P9vbUPwy7i57ZE4vj/W4pPx8J0WxsOB8vP8NwGoRcQ1wGPDfmflwnd6/pyh7BLg3M5/OzDnAY8CimfkS8BFgu4j4FvA1itSmN9fPfaAscHYFDgU6gO/08XpJQ8giRqqeW4F3R8SY2oMRsWxE/D4iRlH82e+uebiTYqqoxwyAzOx5Tm+T6x3luYf3HMjMx4CVKX7ZjwWujIiPzvW6wXr/2ibl1+Z+MCLeDtwNrEBRXB3Zy3l6vNTL8RXKMa1E0UsjNb+OjqH7aBCLGKliMvMZiibesyNiLED5+TRgUmbOAK4A9o+IjogYAewN/O98vtVzFFNBUFwFRfle+1L0m0zMzEPL91pnrtcOxvsPxHrlOI8FJlKkMj1XWs0CuiKiz7+BI2Icxfdzd+BXwE/qME5Jb4JFjFRN+wH3AzeVjau3lvf3LB//ErAkcG/5kcC35/M9vgScGhF3Ulx2/Y/y+E+BLuD+iPgzRX/KyfN47Vt9/4GYCDxVnv8BYHmKomblcry3AX+NiMX6OMeZwGWZOZGiefidEbFfHcYqaT51dHd39/8sSZLUUkbvePaQ/YJ/+aLPN2ROySRGkiS1JNeJkSSpgjraYLE7kxhJktSSTGIkSaqiJgliImJ3iqv7AEZSrNQ9HvgBxVWCEzPz6IjopLiKck2K5RP2nMcaU//GIqbJjFp7fzutG+SOC45gvZ2Oa/Qw2tKLt8/XXpIaRMO7YOZ874SlwTJyWLOUGvWTmecC5wJExKnA2RQLXO4IPAr8PiLWAVYERmbmhhHxfuB7wPZ9ndvpJKm02srLNHoI0pDrrPyv0PbV0dExZB8DUe5kvxrwa2BEZj5SLmh5BfBBYBOKPdrIzFt4Yx2qXlnESJKkoXAEcDTFSt61e7tNo1hPaiwwpeb47Ijoc8bIIkaSpApqpiSmXPn6XeXmqVOB2m1RxgCT53G8MzNn9XVeixhJklRvmwFXAmTmVGBmRKxUbvuxFcXmqzcC2wKUPTH39ndSG3slSaqgJlsnJiiaeHvsQ7EnWRfF1Um3RsTtwIcj4iaKa6v26O+kFjGSJKmuMvOEue7fArx/rmNzKIqbAbOIkSSpgposiakLe2IkSVJLMomRJKmKqh/EmMRIkqTWZBIjSVIF2RMjSZLUpExiJEmqIJMYSZKkJmUSI0lSBZnESJIkNSmTGEmSKsgkRpIkqUmZxEiSVEXVD2JMYiRJUmsyiZEkqYLsiZEkSWpSJjGSJFWQSYwkSVKTsoiRJEktyekkSZIqyOkkSZKkJmUSI0lSFVU/iDGJkSRJrckkRpKkCrInRpIkqUmZxEiSVEEmMZIkSU3KJEaSpAoyiZEkSWpSJjGSJFWQSYwkSVKTMomRJKmKqh/EmMRIkqTWZBIjSVIF2RMjSZLUpExiJEmqIJMYSZKkJmUSI0lSBZnESJIkNSmTGEmSqqj6QYxJjCRJak0mMZIkVZA9MZIkSU3KJEaSpAoyiZEkSWpSJjGSJFWQSYwkSVKTMomRJKmCTGIkSZKalEWMJElqSU4nSZJURdWfTTKJkSRJrckkRpKkCrKxV5IkqUmZxEiSVEEmMZIkSU3KJEaSpApqgyDGJEaSJLUmkxhJkirInhhJkqQmZRIjSVIFNVMQExGHAx8DhgOnAdcC5wLdwH3AhMycExHfBLYDZgEHZeZtfZ3XJEaSJNVNRIwHNgI2BjYHlgNOAo7MzE0pNkjYPiLWKR/fANgFOLW/c1vESJJUQR0dHUP20Y+tgHuBi4FLgcuAdSnSGIDLgQ8BmwATM7M7M58AhkXEEn2d2OkkSZJUT4sDKwAfAd4B/A7ozMzu8vFpwMLAWGBSzet6jj/X24ktYiRJqqAm6omZBDyYmTOBjIhXKKaUeowBJgNTy9tzH++V00mSJKmebgC2joiOiFgGWBD4U9krA7ANcD1wI7BVRHRGxPIUac3zfZ3YJEaSpArq7GyOKCYzL4uIzYDbKMKTCcBjwJkRMRx4ALgwM2dHxPXAzTXP65NFjCRJqqvMPGQehzefx/OOAo4a6HktYiRJqqAm6ompG3tiJElSSzKJkSSpgtw7SZIkqUmZxEiSVEFtEMSYxEiSpNZkEiNJUgXZEyNJktSkTGIkSaogkxhJkqQmZRIjSVIFtUEQYxIjSZJak0WMJElqSU4nSZJUQe3Q2GsRo7YxbFgnZx3zWVZYZlFmz57Dft/6FQ/9/Z8A/NdXPvFvzz1g1w+w01brAvDHG/7KcWdcPuTjlQbb7Nmz2e+Le/HQQ0lXVxdnnHUOzH6VPffam+7ubt773jU56Qc/pKurq9FDlQbEIkZtY+tNVmNYVycf2P0kttjgXRy9/0c54Nu/5qxv7cYqyy/5+vNWXHYxdt72fWy22wl0d8Ofzj6I3119D/f97ZkGjl56635/2aUAXH3djVx37TUcevB/0tXZwTHHHscmm27GXp/fncsu/R3bf3yHBo9Ug6ENghh7YtQ+/vb4vxjW1UlHRwdjFxrJa7Nms+CoEXz79D/wy9/f/vrznvrni2w/4VTmzOmmu7ubBYZ18cqrsxo4cmlwfGz7j3Pq6WcA8MTjj7Pkkktx0UUXscmmmzFz5kz++c9nWXLJpRo8Smng6prERMR4YJ/M3GWQz7sc8D1gSWAU8GfgoMycOZjv82ZFxPLAmpl5aaPHojdMf/lVll9mMe65+OssNm5BdjzwdB5/ZhKPPzOJLTde7fXnzZo1h0mTpwPwnS/vwN0PPsXDT/yrUcOWBtWwYcPYc4/P8btLLuaX519IV1cXDz/yONtt/SEWHrswq0Y0eogaJO3QE9NySUxEdAGXAN/LzPGZuQHwGnBMY0f2b7YANm70IPTvDvjMFlx58wO89+PHsMHO3+HMY3ZjxPB51/Ejhg/j3ON2Z6HRIzjwO+cP8Uil+jrrnPP4y/0Psd8+ezF9+nRWWGEF7nvgb+y59z4c+tX/bPTwpAFrSE9MRHwYOBZ4BZgEfB44Fzg2M++IiAQOy8yLI2IisEdmPl2+fBPgycy8teaUh1IWZBHxFWAXYBZwXWYeGhFHASsDiwOLAqcBOwKrAp8DngXOB54EVgR+DawOrA38PjOPiIg1gJOBjpoxr12+90zgHeU5jgcOA0ZHxE3A28v3mAPckJkHD8K3UG/Ci1NfZtas2QC8MOVlFhjWRVfnvOv4C77/Ra69PfneuVcO5RCluvrlz3/G008/xcGHHs7o0aPp7Oxkhx124KQfnMrKq6zCQmPG0NnLnwm1njYIYujo7u6u28nnNZ0UER3Ao8Ammfl0RBwILAf8BVgB+DnwP8CNwOHApZm5Wc3rPwWsn5lfnsf7rQGcRVHozAIuAs4G1gOWzcy9IuIwYJ3M/I+I2ANYE/hv4HaKQmcU8BiwLPAy8HhmLhURtwCfz8z7I+ILwDuB/6UoiN4LjACeycyFI2J34F2ZeVhE3A4ckJm3RMS+wJmZ2WuDxV8ffqZ7tZWXGfD3WJIGavr06eyxxx48++yzvPbaaxx22GEsscQSHHzwwQwfPpzRo0dz1llnsfTSSzd6qG3hlVkwchh1KzXWO/bq+v2Cn8sdR36gISVTI5KYxYGpNcnKdcBxwLcppomeB74L/CewDTB3X8njFCnK6yJiMWBDigLklsx8rTx+PdDT7HBn+XkycH95+0VgZHn70cycEhGvAv/MzBfKc/T8ELwbOC2K+eIFgIfK4/eWRcmsiJgxj693D+CrEfFd4Gbo+wd2vZ2O6+th1dGMu05h1Nr7N3oYbenF209p9BDaQteIBfnpL3/zb8dGDoM/XXvjvx17xT72SrAnpj6eB8ZGRE+pvznwUGa+SJF87Az8EXgCOAj47VyvvwV4R0SsD68nO0cBmwEPAhtExLDy+Ga8UWz0V5H293gCn83M8cAhwO/7eN0c3vje7kWRRm1OMf20UT/vI0mSBmAokpgtI+KOmvufpvjF/tuImEORhuxePnYJRf/LCxFxBbBfZj5Se7LMnBMROwGnRMSCwIIUhc2RmTkzIn5DMRXVCdxAMTW15iB8HfsCPy0biwG+APQ273Mv8LWIuLO8fXtEPAc8Ddzay2skSRo0bRDE1LcnRvNv1Nr7+z+kQZxOahynkxpn5DCnjxqpnj0x6x93zZD9PrntiPFt0xMjSZLqzJ4YSZKkJmUSI0lSBbVBEGMSI0mSWpNJjCRJFWRPjCRJUpMyiZEkqYLaIIgxiZEkSa3JJEaSpAqyJ0aSJKlJmcRIklRBbRDEmMRIkqTWZBIjSVIF2RMjSZLUpExiJEmqIJMYSZKkJmUSI0lSBbVBEGMSI0mSWpNFjCRJaklOJ0mSVEE29kqSJDUpkxhJkiqoDYIYkxhJktSaTGIkSaoge2IkSZKalEmMJEkV1AZBjEmMJElqTSYxkiRVUGcbRDEmMZIkqSWZxEiSVEFtEMSYxEiSpNZkEiNJUgW5TowkSVKTMomRJKmCOqsfxJjESJKk1mQSI0lSBTVTT0xE3AVMKe8+BvwY+AEwC5iYmUdHRCdwGrAm8CqwZ2Y+3Nd5LWIkSVLdRMRIgMwcX3PsbmBH4FHg9xGxDrAiMDIzN4yI9wPfA7bv69wWMZIkVVATBTFrAqMjYiJF3XEUMCIzHwGIiCuADwJLA38EyMxbImK9/k5sT4wkSaqnl4ETga2AfYBzymM9pgELA2N5Y8oJYHZE9Bm2mMRIklRBHTRNFPMQ8HBmdgMPRcQUYNGax8cAk4HR5e0enZk5q68Tm8RIkqR6+jxFfwsRsQxFsTI9IlaKiA6KhOZ64EZg2/J57wfu7e/EJjGSJFVQE60T8xPg3Ii4AeimKGrmAL8AuiiuTro1Im4HPhwRNwEdwB79ndgiRpIk1U1mzgQ+PY+H3j/X8+ZQ9MwMmEWMJEkV1EzrxNSLPTGSJKklmcRIklRBbRDEmMRIkqTWZBIjSVIFdbZBFGMSI0mSWpJFjCRJaklOJ0mSVEFtMJtkEiNJklqTSYwkSRXkYneSJElNyiRGkqQKaoMgxiRGkiS1JpMYSZIqyMXuJEmSmpRJjCRJFVT9HMYkRpIktSiTGEmSKsh1YiRJkpqUSYwkSRXUWf0gxiRGkiS1JpMYSZIqyJ4YSZKkJmUSI0lSBbVBENN7ERMR3+jrhZl5zOAPR5IkaWD6SmLaoIaTJKma2qEnptciJjOP7rkdEQsCKwH3AaMyc/oQjE2SJKlX/Tb2RsQWwD3AJcCSwOMRsWW9ByZJkt68zo6h+2jY1ziA53wH2ASYnJnPApsBJ9R1VJIkSf0YSBHTWRYvAGTm/XUcjyRJGgQdHR1D9tEoA7nE+qmI+AjQHRHjgAnAE/UdliRJUt8GksR8EdgVWA54FFgL2Lueg5IkSW9NxxB+NEq/SUxm/gv4VESMBWZl5sv1H5YkSVLf+i1iImIN4DxgeaAjIh4APpeZj9R7cJIk6c3pbIN1YgYynXQ68LXMXDwzFwO+B5xd32FJkiT1bSBFzKjMvLznTmZeDIyt35AkSZL619feScuXN++JiMOAnwCzKJp8rx+CsUmSpDepDWaT+uyJuRbopmg8Hk9xlVKPbuBL9RuWJElS3/raO+kdQzkQSZI0eNp6A8geEbEKsD+wEEUq0wW8IzM3q/PYJEmSejWQxt5fAZOBtYG7KS61vq+eg5IkSW9NR8fQfTTKQIqY4Zn5TeCPwJ3AtsDmdR2VJElSPwZSxLwcESOAh4B1M3NGncckSZLeos6OjiH7aJSBbAD5c+BSikurb46IrYGn6zoqSZKkfvSbxGTmKcCOmfkcxaXWZwAfr/O4JEnSW9AOPTF9LXb3jbnu195dAzimTmOSJEnqV1/TSdW/wFySpIpqh3ViOrq7uxs9BtWYPGO2/0MaZNyoLibPmN3oYbSl6a/6fW+UZccN5+nJMxs9jLa17Ljhdas0Jlz8wJD9Pjl1h3c3pGIaSGOvJElqMQO5/LjVtcPXKEmSKmhASUxELAisBNwLjM7M6XUdlSRJekvaoSem3yQmIj4I3ANcAiwFPB4RW9Z7YJIkSX0ZyHTSccAmwOTMfBbYDDihrqOSJElvSWfH0H007GscyHPK4gWAzLy/juORJEkakIH0xDwVER8BuiNiHDABeKK+w5IkSW9FIxOSoTKQJOaLFPsmLQc8CqwF7F3PQUmSJPWn3yQmM/8FfGoIxiJJkgZJO1yd1G8RExGPAf9v1b/MfGddRiRJkjQAA+mJGV9zewFgB2BEXUYjSZIGRTv0xAxkOunxuQ6dEBF3AMfWZ0iSJEn9G8h00mY1dzuA1YBRdRuRJEl6y5qtJSYilgT+DHwYmAWcS9Guch8wITPnRMQ3ge3Kxw/KzNv6OudAppOOrrndDTwPfG6+Ry9JktpSRCwA/BiYUR46CTgyM6+JiNOB7SPicWBzYAOKK6IvAt7X13kHUsScn5mnv+mRS5KkIdfZXFHMicDpwOHl/XWBa8vblwNbAglMzMxu4ImIGBYRS2Tmc72ddCDrxOz/5scsSZLaWUTsDjyXmVfUHO4oixWAacDCwFhgSs1zeo73aiBJzJMRcRVwK2/EQGTmMQN4rSRJaoCBpBRD5PMUq/5/iGLB3J8CS9Y8PgaYDEwtb899vFcDKWJuqbndVNmUJElqbpn5+gVCEXENsA/Flc7jM/MaYBvgauBh4L8i4kTg7RR7Nz7f17l7LWIi4nOZeV5mHt3bcyRJkt6ErwBnRsRw4AHgwsycHRHXAzdTBEkT+jtJX0nMgcB5gzFSSZI0tJqrr7eQmeNr7m4+j8ePAo4a6PmaaMpMkiRp4PpKYlaLiEfncbwD6HbvJEmSmleTXWJdF30VMQ8D2w7VQCRJkuZHX0XMzHnsmyRJklpAGwQxffbE3Dhko5AkSZpPvSYxmelKvZIktajONk9iJEmSmtZAVuyVJEktph2uTjKJkSRJLckkRpKkCmqDIMYkRpIktSaTGEmSKsirkyRJkpqUSYwkSRXUQfWjGJMYSZLUkkxiJEmqIHtiJEmSmpRJjCRJFWQSI0mS1KRMYiRJqqCONliy1yRGkiS1JJMYSZIqyJ4YSZKkJmUSI0lSBbVBS4xJjCRJak0mMZIkVVBnG0QxJjGSJKklmcRIklRBXp0kSZLUpCxiJElSS3I6SZKkCmqDvl6TGEmS1JpMYiRJqqBOqh/FmMRIkqSWZBIjSVIF2RMjSZLUpExiJEmqIBe7kyRJalImMZIkVZAbQEqSJDUpkxhJkiqoDYIYkxhJktSaTGIkSaoge2IkSZKalEmMJEkV1AZBjEmMJElqTSYxkiRVUDukFO3wNUqSpAoyiZEkqYI62qApxiRGkiS1JJMYSZIqqPo5jEmMJElqUSYxkiRVkCv2SpIkNSmTGEmSKqj6OYxJjCRJalEmMZIkVVAbtMSYxEiSpNZkEiNJUgW1w4q9FjGSJKluIqILOBMIYDawB0Xf8blAN3AfMCEz50TEN4HtgFnAQZl5W1/ndjpJkiTV00cBMnNj4BvASeXHkZm5KUVBs31ErANsDmwA7AKc2t+JLWIkSaqgziH86Etm/g+wd3l3BeCfwLrAteWxy4EPAZsAEzOzOzOfAIZFxBL9fY2SJEl1k5mzIuI84IfAhUBHZnaXD08DFgbGAlNqXtZzvFcWMZIkVVBHR8eQfQxEZn4OWJWiP2ZUzUNjgMnA1PL23Md7ZREjSZLqJiJ2i4jDy7svA3OAOyJifHlsG+B64EZgq4jojIjlgc7MfL6vc3t1kiRJFdREF1j/FjgnIq4DFgAOAh4AzoyI4eXtCzNzdkRcD9xMEbJM6O/EHd3d3f09R0No8ozZ/g9pkHGjupg8Y3ajh9GWpr/q971Rlh03nKcnz2z0MNrWsuOG163WuODuZ4bs98lOay3TkJrJJEaSpApqh8Xu7IlR2zrphOPZcvzGjN9ofX527tncf//9bP3Bzdhqi035yoH7M3u26YCq4847buOTH/kwAI89+jA7bP0BPrHNFuy7777MmTPn9efNePllttz0fVx95RWNGqo0YBYxaks3XHcNt91yM3+86noum3gVTz/1JEcccQRfP/pYrrjqembMeJnLL7u00cOUBsVpPziRg7+0D6+++goAx3ztEA4+8ih+e/lVdHd3c8Uf3vhZ/9rBB7bFv+DbQbOsE1NPFjFqS3+6ciLvWW11PrPzjnzqk9uz1TbbcdFFF7HxJpsxc+ZM/vXssyyx1JKNHqY0KFZ4x0qc+bPzX7//l3vuYsONNwNgm2224YZr/gTA6T88iXXXfz/vWf29DRmnNL/q0hNTXja1T2buUnPseOBB4G7gY5l5TC+v3R14V2Ye1sf5FwFOBFYBuoAngS9m5pTeXjOUImJRYOvM/GWjx6J5e+H5STz55OP8+qLf8fjfH+PTO+3AQ/kgTzzxGDtstxVjxy7MKqtEo4cpDYrtPrYDTz7x9zcOdHe/nraMGTOGqVOncsO1V/HYIw/z3f8+jTtuvbkxA9WgaodEbcgbezPzbopC5q34FfDjzLwYICK+DPyYYq+FZvBe4GOARUyTWmSxRVklguHDh7PKqsGIkSN57rnnWH75FfjzvQ/y03N+wtcO+yo/OvOcRg9VGnQdnW+E8NOmTWPswgvzq5+dy9NPPsEnP/JhHvlbcu89d7HkUm9jtTXWbOBIpb4NeRFTm9JExBeA/YEXgJlAT975/oiYCCwB/Cgzz6h5/QrA23oKmNLJwELl47tSXIP+KvA3iv0adqXYgGoUsDTwA2B7YHXgq5l5SUQ8DNxEke5cRbHU8fpAZuZuEbEccAYwEnilPG8XRUH1JLAScFtm7gt8DVgzIvYGngcOBV4D/g58NjPf6KJTQ7x/w4358Wk/ZMKXvsyz//gHL0+fzhe+8AWOOu4EVlp5FRYaM4bODmdbVU2rr7EmN91wLRttsjmXX345G2y4KR/7xE6vP/7l/fbkY5/YyQKmxVU/h6lvEbNFRFxTc/+dFLtXAhARi1P8cl+LouC4uua5rwFbUWwU9QeK4qHHMsBjtW+UmbOBKRGxGHA0sHZmTouI7wNfBF4CxmTmlhGxC/Bl4P3AeOBA4BJgRWAL4B8URdUGwAHAoxExjmL66uTMvDwiPggcT1GsrApsSbEK4aMR8Tbg2xSF2hkRcQHw/cz8dUR8lmJviD6XUVb9bb3tR7jpxuv54KYbMmfOHE74/sksvfg49tv78wwfPpxRo0Zz8mln9H8iqQV949jvcvCB+3H8a19nzdXfw3bbf6LRQ5LelHoWMVd5yjg3AAAWqklEQVTNoyem1srA/Zn5cvn4TTWP3ZmZ3RHxLDB6rtc9Aby99kBELADsRJG8/DUzp5UPXUdRYNwK3FUemww8UJ7/RYpkBWBSuWsmETE9M+8vb08pn7MGcEREHEpR4PasDvVwz/tFxD9qztfjP4HDI2JfilUJ/+f/f6veMGZEJ12d7VA/N97JJ534/47devNN83im6m3cqK5GD6Hylh23Knfefmtxe73VueXG63p97m9++dOhGlZbq/cig23QEtPQxe4eBt4VEaMokpj1KRp/AXpdZTAzn46I5yNi+8y8pDx8YPn6/YD3RMSCmTkd2Bx4qL9zDvDxB4ETM/OmiHhXee7eXjeHN6782hs4KjP/FRE/BnYAzuvtTaa96kxTo7hib+O4Ym/juGKvWlnDipjMfD4ivkux6dMLFP0qr1Hsq9Cf3YBTI+KrwHDgEWCvzJwSEd8Ero6IORSF0mEMTsPvV4EfRcTIcqwH9vHcR4A1IuIg4DbgfyNiEsW24pcNwlgkSepTZxt0xTRs76SIGAYcmpnfLu9fBxyZmb1nnG3AvZMaxySmcUxiGsckprHquXfSpff+c8h+n3x0jaXaa++kzJwVEQtGxJ0U/SW3UqQykiTpLbInps4y8wjgiEaOQZIktSZ3sZYkqYI62qAnxtW8JElSSzKJkSSpgtqhJ8YkRpIktSSTGEmSKqgd1okxiZEkSS3JJEaSpAqyJ0aSJKlJmcRIklRBJjGSJElNyiJGkiS1JKeTJEmqILcdkCRJalImMZIkVVBn9YMYkxhJktSaTGIkSaoge2IkSZKalEmMJEkV5GJ3kiRJTcokRpKkCrInRpIkqUmZxEiSVEGuEyNJktSkTGIkSaoge2IkSZKalEmMJEkV5DoxkiRJTcokRpKkCmqDIMYkRpIktSaTGEmSKqizDZpiTGIkSVJLMomRJKmCqp/DmMRIkqQWZRIjSVIVtUEUYxIjSZJakkmMJEkV5N5JkiRJTcokRpKkCmqDZWJMYiRJUmsyiZEkqYLaIIgxiZEkSa3JIkaSJLUkp5MkSaqiNphPMomRJEktySRGkqQKcrE7SZKkJmUSI0lSBTXLYncRsQBwNrAiMAI4FrgfOBfoBu4DJmTmnIj4JrAdMAs4KDNv6+vcJjGSJKmePgNMysxNgW2AU4CTgCPLYx3A9hGxDrA5sAGwC3Bqfye2iJEkqYI6hvCjHxcAX6+5PwtYF7i2vH858CFgE2BiZnZn5hPAsIhYoq8TW8RIkqS6ycyXMnNaRIwBLgSOBDoys7t8yjRgYWAsMKXmpT3He2URI0lSFTVRFBMRywFXAz/LzF8Cc2oeHgNMBqaWt+c+3iuLGEmSVDcRsRQwETg0M88uD98VEePL29sA1wM3AltFRGdELA90ZubzfZ3bq5MkSaqgJlon5ghgEeDrEdHTG3MgcHJEDAceAC7MzNkRcT1wM0XIMqG/E3d0d3f39xwNockzZvs/pEHGjepi8ozZjR5GW5r+qt/3Rll23HCenjyz0cNoW8uOG163SuOux6cN2e+TtVcY05CKySRGkqQKapZ1YurJnhhJktSSTGIkSaqgNghiTGIkSVJrMomRJKmK2iCKMYmRJEktySRGkqQKaqJ1YurGJEaSJLUkkxhJkirIdWIkSZKalEmMJEkV1AZBjEmMJElqTSYxkiRVURtEMSYxkiSpJZnESJJUQa4TI0mS1KRMYiRJqiDXiZEkSWpSFjGSJKklOZ0kSVIFtcFskkmMJElqTSYxkiRVURtEMSYxkiSpJZnESJJUQS52J0mS1KRMYiRJqiAXu5MkSWpSJjGSJFVQGwQxJjGSJKk1mcRIklRFbRDFmMRIkqSWZBIjSVIFuU6MJElSkzKJkSSpglwnRpIkqUmZxDSZcaO62qB2bl7jRnU1eghtye97Yy07bnijh6A6aIdfJiYxkiSpJZnESJJURW0QxZjESJKklmQSI0lSBblOjCRJUpMyiZEkqYJcJ0aSJKlJWcRIdRARneXnNvi3kPSGiHDRnybRMYQfjWIRIw2yiOjMzDkR8TbgwIhYutFjkoZCRHRl5uyI6IyIfSJikUaPSdXW0d3d3egxSJVTFi7/A5wLvApcCkzPzJcbOS6p3soU8iIggbuAywEyc2ojx9WO/j7plSH7Bb/iYiMbEsiYxEiDJCLWjoieZvmdgMsoCpkdgVOArRs1NqmeIuLdNXd3BqYCxwD/AXwL2KER41L1mcRIgyAiRgLbAesB9wLLAOsC04GvAZ8DFgcOzUz/0Kkyyh6Y/wDWBu4GxgHvA2YB5wMrA+8FJvizP7Qen/TqkH2/V1hshEmM1Koy8xVgDnAIsGVmnpiZnwJuAzYGPgmc4V/iqprMnA08AOwO7JyZpwF7UkwjTQY+DVzgz77qwSJGegvmuhLjd8C+wN8j4rDy2AxgVWCPzHx4qMcn1ctcP/sPACcAj0TEcWVhMw7YFjgxM69uxBhVfU4nSW9SzVVIywJfp2jc/UpErAXsCiwPXAOcm5kzGjhUaVDVXoUEHA88A/wIWAHYHxgL/AU4KzOnRkSHSczQe+KFoZtOWn7RxkwnWcRIb0FELA6cAVxJ0RPTmZnbRMQKwF7ATzPzoUaOUaqHcg2kHwEvA2sCDwFHUCQwXwEuzsw/NW6EsoiR1KuIGAEcB7wjMz9RHjsfWD4zN2zo4KQ6i4j/AlbLzO3K+78AXgEO6FlKwASmsZ4cwiJmuQYVMfbESPOhZiXersx8FbgWWCQivgiQmTsDWSYxUmXMYyXeicAaEfHZ8v5nKVKYd/Y8wQJG9WYSIw1QTQ/MMsB3gRuBBYD7KS4xvS8zf9jIMUr1UPOz3wmcBDwITKNY0O444NeZeXYjx6j/76kXhy6JefsiTidJTS8ilgB+C5xKsRbMh4GfUMToH6O4xHqK/wJV1ZQ9MOcDtwN/B44EDqX42T8e+ATwz/LKJDWBdihihvX/FKm9zTWvvxTFpdRXARdQFDOzgOuBqzNzemNGKQ2+ngSm5y7wJEXRfjZwGrAExRYD22XmpMaMUr2r/v6zJjFSH2pi9NHAQsBI4AZgEsV2AqsChwMfdW8YVUnPZdTl7TEUP/s/o5hCPZlibZizgF0y85mGDVS9eurFmUOYxAy3sVdqJmUC07MOzMUUmzguSbEa6VLAFsBRwL4WMKqS8me/ZzfqX1DsAbYoRQq5GDCcooD5rgVM8+roGLqPhn2NJjHS/1ezmNcIio3sbgVeAs4DtgSmAG8Hns3MRxs3UmlwzZXAHE4xXfoCsDewG8Uijl3AjMy8rmEDVb+enjx0Scyy4xqTxNgTI82lpoBZFjiYYmO7UzLzyYjYD7iDogfgyoYOVBpkc63EexiwIbBPZj4dEcMp+l/2zsybGzpQDUizdcRExAYU6d34iFgZOBfoBu6j2CB0TkR8k2Lh0FnAQZl5W1/ndDpJmkv5l/gSwHcolk6/E/hDRCyTmRcDOwBPNHKMUj2UP/sdFD/7C1Ckj78tH/sRRSO7vzc03yLiEIopyJHloZOAIzNzU4p6a/uIWAfYHNgA2IXi561P/jBKpfIv757P+wIbZubZmfllin+B3hoRS2fmH9xKQFXSs4hj6WiKZt2jy53YH4qIuwAy8/TMvLEhg9R8a7KemEcoLsPvsS7FYqFQ7Hj+IWATYGJmdmfmE8Cw8h+UvbKIkXj9KqTuiBhLseroT4C/RMQ5AJl5FHA6MKpxo5QGXzmFNKe8PRz4FvBCRJwNkJm7URQybqWhNy0zLwJeqzlUu3TFNGBhio1Dp9Q8p+d4r+yJUduruQppaeAKip6XR4CDgEMi4oLM3Ckzv93QgUqDrPYqJOAPwIsUi9mtB1wREb/IzF3L7TTUYjqarivm38ypuT0GmAxMLW/PfbxXJjFqazUJzCiKq46+TzFXuxiwK/A94MmywJEqo0xgev4lvAtwC/B14H3AgcBWwIoRscZc003SYLgrIsaXt7ehWDD0RmCr8tL+5YHOzHy+r5OYxKht1SxktzTFCqTTgEsy876I+BXFejA7Aodk5qxGjlUaTOXPfk8CcxZFE++fMvPhiPg6cCKwQGZu3NCB6q1p6iCGrwBnllOYDwAXlj+T1wM3U4QsE/o7ievEqK1FxDiK5OUOYCawOvCHzJwYEWsDz2TmPxs5Rqkeygb2HwDTKTZ03A74WWZeWl7+ulBm3t3IMeqteXbqa0P2C/5tYxdwnRhpKPTshVT+K3R/io0c/wj8A/gUsFNEzHEdGFXNXPuA7QasD3w2Mx+KiOnA7hGxQGb+tnGj1GBp7iBmcDjPqbbS0wcQEQtSROhXU6wD8wlgceA3FHOz9zZulNLgq+2BKVOYBC4D9omIlTLzQoqf/0caOExpvjidpLZRk8C8nWIDOyg2cxwOjCjvnwn8o+Zfq1LLmyt9PIMihX+Mog9sOvBu4GS30KiWf00buumkJcc0ZjrJJEZto2YdmLOB84Ejyoe6KRrJuoBXLWBUNeXP/jDgAorVpn9Yfl4eeJri0uoFGjdC6c2xJ0aVVsbmG1EUKH8FFgJmZeb55eOzgP8GTgGu8iokVUlErE7xs/83iiJlTmYeUz42CfgARQH/v5n5asMGqrpo8nViBoVJjCqrjM4vAL4MfAYYkZmPA4+Wl5ECrEAxlTTKAkZVEhEnAd8FDgFWy8wpQGdEnFg+ZRiwErC4BYxalT0xqqQygTkPeDgzj4mIhTLzpYgYQ7HB2EbAOhT9MF/KzPsaOFxpUEXE8cCiwD7A6Mx8qTy+BsVCdstSLOh4TGZe1rCBqq6ee2nWkP2CX2KhYV5iLQ2ihYFJPdE5MKP8vBTFZaXfpbi0+l+ZOakB45Pqouz7WgT4SrmY48vl8VUoVqHeH3gXMNVGXrU6ixhVTjmNtATwoYhYLjOf5I0lE5YGli+j9Sm9nUNqRWUCOZqiYXd4ebiTYp+aWcCKmfkK4CJ2baD6HTH2xKiCMnNOZv4NuBiYEBGL1PS7LAOMjojR5V/4UmVkZndmPgs8CpxQrg3T87P/dmDRiFjUvZBUFfbEqHJq1sR4H/BRivn/H1OshbE7MCEz72/gEKW6qNkPbBmKht6lgG8CK1Js7nhsZl7RwCFqCE2aPnQ9MYst2JieGIsYVUpEDKu9yigilgN2oihguoGTMvPBRo1PGmxl2jJ7HscXAr4BjAMWBM5xK432YhEjtZCIGEWxiNfxmflwo8cjDZWI6ALe0fNzP3dhU+6H9FrDBqiGaIcixnlRtbSeuf0ygZkB3AMcFRGjy+P2vaiSIuLTEbF1efci4HsRcRlAZs6OiOE1T3cNpDbUMYT/NYpFjFpaOf+/MHBRREwA/gD8CViufIpFjConIsZRXIG3dURcANyWmdsDUyLiUoDMnNnzfLfSUFVZxKgl1V5dUV4uPRPYA/gFxVoYu5ePzWnE+KR6KaeKJlOsRv0XijWRngXIzF2B1yLifxs4RDWJjo6h+2jY12hPjFpVRCwC7EuRvLwCvI2iefczFMupfxR40X+Fqip6el3KIn4Xih2olwTWACb2rL4bEWtlpmvBtLkXX549ZH/3LTK6y54YqT81PTAjKdKXxSlW4P0h8EVgZmZ+Ftg2M1+wgFFVlJdP9xQwPwYOBzYBngfuBLaJiI8DWMCoXVjEqGXUrIGxNEWUfijFrtSXUBQxSwPfjIjR5RSTVAnlytM9U6PnUixm9zGKQn5tYBJwF/BAQwYoNYhFjFpGWcCMA35LUbhcCDwGHJmZFwCfBXbKzJcbOExpUJWLNm5Wc2hB4KLMfAz4ATCeYjPTmzIzvSJPPdqhJ8a9k9RqFgUeyMyzACLiVeDLAOVWA1JlRMQHKPq6fhERhwCXA38EjouIAyj+Dn8N2AJ4CrjfKVS1E5MYtZopwOYRsXd5/73AqmWTr1QZEbEqsBuwSUSsDbwKHAY8A1wF/Jriarz/AM4B3hURXSYx6tEO68R4dZKaVkQcBdySmX8s7w/LzFkRsR7wG2Ai8D5gV7cSUBVFxCbAVsC/gBuB1YEPA98BHqFYVuA5iu0F/iMzs0FDVROaMmPOkP2CX3hUp1cnST0iYgTF2hefLSN1gDnlJaZ3UPQBfB/YzgJGVRIR7+q5nZk3AKOBPYENgYeAG4CjgDlAUlydt4sFjObWDj0xJjFqWuVeSJ8EPgKckZl/Ko/vBSwDfMvF7FQlEbERcDTw7cy8JiJ2B3YAfk6xFswLwC3AU5n5VPka90XSPE19ZeiSmLEjG5PEWMSoqZWFzE7AdsC3gLdT/Ct0r8y8t4FDkwZduefXjsAHgReBFYH9M/PpiPgwxZ+DkzLziYjosIlXfZk2hEXMGIsYad7KQuYTwCEUl5d+xCkkVVX5874LRZ/L4Zn565o1khbJzBcbPES1CIsYqUnUJDK3WcCo6sqesF0oer/O72lul+bHtFeHsIgZYREj9cn4XO2kLNw/DWwD7J2ZLzR4SGoxFjGSpIYpC5lFM/PpRo9FreelV4fuF/xCIxpzjZJFjCRJFdQORYzbDkiSVEGNXL9lqLjYnSRJakkmMZIkVVAbBDEmMZIkqTWZxEhtKCJWpNiH536gGxhOsTvyHj3L2b+Jc+4OjM/M3SPiD8CemflML889GrgyM6+fj/N3Z2bHXMeOAsjMo/p43d/Lcf19gO/T7zmlltAGUYxFjNS+nsnMtXruRMT3gBOAT73VE2fmtv08ZXPg6rf6PpLam0WMpB5XA9+B19OLW4G1gE2BrYGDKKag/wxMyMxXImI34EhgKvA48FLN68dT7ER+KrAJ8BrF/lcjgPWAsyJiB2AG8CNgMeBl4IDMvKtMi34OLESx6WGfImJ/YDeKrSlmAp+q2dn5qIhYE3gF+GJm/iUilgJ+DCxHsSP04Zl55Xx9x6Qm1tEGUYw9MZKIiAUodgy/uebw5ZkZwBLAXsBGZXLzL+CrEbEM8F/AZsCGwJh5nPoAiiLk3cCHKPYD+jVwB8V0073AecAhmbkOsHf5OMApwLnle97Yz/jHAh+nmDZaHbgM2L/mKX/LzLUpiqjzymM/AM7OzHWBjwE/joh5fQ2SmpRJjNS+lomIu8vbI4DbgMNqHr+1/PwBYBXgloiAon/mTmAj4KbM/CdARPycYvflWpsDZ2TmHIpUZrXyuZSfFwLeB5zTcwxYKCIWo0hyeqa2fgH8pLcvJDOnRsSngV0iYlWK5OjumqecVT7vDxHx84gYR1FUvSsijimfswCwUm/vIbWadlgnxiJGal//1hMzDzPKz13AbzLzS/B64TGMomCp/Wty1jzO8RpF4zDla1cGnqh5vAt4Za7enLcDL5Sv60mLu4HZvQ00IpYDrqFIby6nKJjW7mVsHeW4uoAtevYkioilKVKmj/f2PpKai9NJkvpzDbBDRCwZER0U/SsHATcAG0bEshHRCew8j9deB+wcER0RsSRwLUXqMwsYlplTgL9FxGcAIuLD5WsArgQ+U97+BDCyjzG+D3g4M78P3A7sQFGk9Ni1PP8OwAOZOR24CtivPP4e4D5g9MC+JVLzGzmMjqH6aNTXaBEjqU+ZeQ9wNMUv/b9SFAfHl9NIB1AUG7dRNPfO7TRgOnBP+bwDMnMa8Efg9IjYiKLA2DMi/kLRWLxzuVv5/sCOEXEPsC0wrY9hTgQ6I+J+iqmuB4F31Dy+ajl19p/A58pjBwDvL9/3fOAz5dgktQg3gJQkSS3JJEaSJLUkixhJktSSLGIkSVJLsoiRJEktySJGkiS1JIsYSZLUkixiJElSS7KIkSRJLen/ACtNMu+Vy7rQAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 576x576 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAi4AAAIjCAYAAAAk1JABAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3XecXHW9//HXbgppJKFIrxL5oHQNSPsBFrgU4YpcBEWa0gmKyJVmoVjwIhZEVEAUy1WKIoIURUSatIs0gQ8GRBAISICEBBJIdn9/nLNhCNnNQnbmZE5eTx7z2Dllznxndtj55P39nu/p6O7uRpIkqR10Vt0ASZKk/rJwkSRJbcPCRZIktQ0LF0mS1DYsXCRJUtuwcJEkSW1jcNUNkCRJA2/4hhNaNt/JS389o6NVz2XiIkmS2oaJiyRJddRRz2yinq9KkiTVkomLJEl11NGyYSctZeIiSZLahoWLJElqG3YVSZJURw7OlSRJqpaJiyRJdeTgXEmSpGqZuEiSVEeOcZEkSaqWiYskSXXkGBdJkqRqmbhIklRHjnGRJEmqlomLJEl15BgXSZKkapm4SJJUR45xkSRJqpaJiyRJdeQYF0mSpGqZuEiSVEeOcZEkSaqWiYskSXXkGBdJkqRqmbhIklRHjnGRJEmqlomLJEl1ZOIiSZJULRMXSZLqqNOziiRJkipl4iJJUh05xkWSJKlaJi6SJNWRM+dKkiRVy8JFkiS1DbuKJEmqIwfnSpIkVcvERZKkOnJwriRJUrVMXCRJqiPHuEiSJFXLxEWSpDpyjIskSVK1TFwkSaojx7hIkiRVy8RFkqQ6coyLJElStUxcJEmqI8e4SJIkVcvERZKkOnKMiyRJUrVMXCRJqiPHuEiSJFXLxEWSpDoycZEkSaqWiYskSXXkWUWSJEnVMnGRJKmOHOMi6Y2IiNUiojsiPjHX+qMi4scVtGffiLisvH9ORLx/gI7bktcTEcdHxKMRce6bfPydETF2oNv1ZkXE2RHxrl62DdjvR6obExepubqA0yLihszMqhvTIzP3r7oNb8IngI9m5g1v5sGZucEAt2dBbQP8YF4b2vT3o4VNTce4WLhIzfUScBrwvxGxaWa+3LgxIsYA3wU2ALqBK4DjMnNWRMwELgHWB/YEbgC+AbwfGAWcAOwGrAs8AeyUmdMj4uPAQcBQYEnglMz83lzPey1wBjAb+GLDpnHAxZm5V0RsBnwNGFnud2JmXhYRQ4DTKb54nwaeAqbM68VHxLHAPsAs4O/Avpk5JSI+D3ykXP8gMCEzJ5Xt+guwObAKcDVwIPALYCXghxHxBeAQ4IzMvKjx9WTmRRFxIrAL8DIwuXzOJyOiG3hLZj7zRp8/M7vm8f79H7AJsAxwFrAcsFX5fn04M++JiE2A/wEWA5YH/pCZn4iILwMrAD+PiL3L9/lZYC3ge8Cu5e9nMnAhxWfgSeCPwDWZefK83m9pUWBXkdR8XwamAV+Zx7bTKb6c1gXGU3xBHVVuGwpcmpmRmbdTfPlNysyNgfOAc4AjgHcAY4D/jIhRwAHADpm5IbA7xRfnPGXmxZm5QZlGfB6YBBwVEUsAPwL2ysx3Av8JfC8iVgEOBdYsn3cbii/414mInYF9gU0zcx3gH8CEiNgP2B7YKDPXA+4Fftzw0DWArYH1yv22yszdKYqzPTPz/N5eT0SsXL4nG2XmeOD3wLvn2ucNP38vT7daZm4OfIziPb62fM4rgcPLfT4FfCEz303xfu0cEe/KzOMbXs8t5b7PZeY7MvM7PU+QmX+iSGXOBj4HzKT4PEnz19HZulsLWbhITVb+a/1jwH4Rsc1cm7enSAq6M3Mm8P1yXY/r59r/V+XPh4B7MvPx8vj/AJbMzGnAB4AdI+Jk4HiKdKZPZTLwPYrU5ilgU4qE4DcRcSdwOUUitB5F4vO/mflyZk4Hft7LYd8PXJiZz5Xvw5GZ+eXy9f2ofCzAt4H3RcTQcvnSzOzKzKnARIrUqL8eB+4C7oiIrwN3ZuZv5tpnoJ7/1+XPh8qfVzYs9zxmH2BsRBwHnAkMp/ffx9y/6x5fLI93KPCxudMfaVFj4SK1QGY+RtF9cx6wdMOmToqCoHF5SMPytLkONbPh/itzP09ErATcCaxK0bX0ufm1LSLWpCiIPpaZ95erBwH396QxZSKzCXBVub2x83xWL4eeRcNri4ixEbFaeey5X/PghmO+1LCte67n6m39UJhTJG5FkfRMBr4ZEXMnTgPx/PDa3wWZ+brfB3AdsAPwAHASRWHV2/Hm/l33GENRRHYBb+tlH+n1Ojpad2shCxepRcrxGFdQdGX0uIqi+6QjIhajGM/xhwV4mvHAv4EvUXSTfAAgIgbNa+eIWK5s039n5rUNm24G3hYRW5b7bUAxRmXFcv+9I2JYRAyj6I6al6uBD0XE6HL5BOBIimTi4xExslz/SeC6MnHqr3+Xr5WIeAdFEkRErE/R9XN/Zn4V+Caw0VyPHYjnn6/yDKaNgKMz89cUY3TGURROUBR2Q3p5eKMfAj8F9qMYEzNmINsptRsH50qt9Ulgi7mWvwPcQ5EaXMmCjWH4PfBxICn+hf5nii/5cb3sfyLF4NIjIuKz5bonMnOHiNgVOLUsTjopxrs8EhE/KI93L0Wq8fd5HTgzLy+LihsjAuBvFONvpgMrA7dGRCdFd8yeb/B1fgk4LyJ2pEgzriuf866IuAC4PSKmUaQnn5zrsT8cgOefr8x8PiK+StFtNR34F3AjxXv3R4qupp9FxCG9HSMiDqMYQ7RbZr4SEVdRDATurViUaq+ju7t7/ntJkqS2MmLXc1v2Bf/irz7esv4iu4okSVLbsKtIkqQa6qjpBHQmLpIkqW2YuEiSVEf1DFwsXBY2wzec4Gjpitx+4XGM321ek9uq2Z677Yyqm7DIGjoIXp5ddSsWXcMG17W8aB4LF6m09rgVqm6C1HKdfm3WlmNcJEmSKmbiIklSDZm4SJIkVczERZKkGjJxkSRJqpiJiyRJNWTiIkmSVDETF0mS6qiegYuJiyRJah8mLpIk1ZBjXCRJkipm4iJJUg2ZuEiSJFXMxEWSpBpaWBKXiOgEzgTWB2YC+2fmxIbtRwEfAbqAr2TmxX0dz8RFkiQ10weBYZm5KXAMcFrPhogYC3wS2BTYFvjW/A5m4SJJUg11dHS07DYfWwBXAmTmzcD4hm3TgX8CI8tb1/wOZuEiSZKaaTQwpWF5dkQ0DlV5DLgPuAM4fX4Hs3CRJKmOOlp469tUYPGG5c7MnFXe3x5YHlgdWAX4YERs3NfBLFwkSVIz3QjsABARmwD3NGx7DngJmJmZM4DngbF9HcyziiRJqqGF5awi4GJgm4i4iSKf2S8ijgQmZuZvI+L9wM0R0QXcAPyhr4NZuEiSpKbJzC7g4LlWP9Cw/YvAF/t7PAsXSZJqaCFKXAaUY1wkSVLbsHCRJEltw64iSZJqyK4iSZKkipm4SJJUR/UMXExcJElS+zBxkSSphhzjIkmSVDETF0mSasjERZIkqWImLpIk1ZCJiyRJUsVMXCRJqiETF0mSpIqZuEiSVEf1DFxMXCRJUvswcZEkqYYc4yJJklQxExdJkmrIxEWSJKliJi6SJNWQiYskSVLFTFwkSaqjegYuJi6SJKl9mLhIklRDjnGRJEmqmImLJEk1ZOIiSZJUMRMXSZJqyMRFkiSpYiYukiTVkImLJElSxSxcJElS27CrSJKkOqpnT5GJiyRJah8mLpIk1ZCDcyVJkipm4iJJUg2ZuEiSJFXMxEWSpBqqaeBi4iJJktqHiYskSTXkGBdJkqSKmbhIklRDNQ1cTFwkSVL7MHGRJKmGHOMiSZJUMRMXSZJqqKaBi4mLJElqHyYukiTVUGdnPSMXExdJktQ2TFwkSaohx7hIkiRVzMRFkqQach4XSZKkipm4SJJUQzUNXExcJElS+zBxkSSphhzjIkmSVDETF0mSasjERZIkqWImLpIk1VBNAxcTF0mS1D4sXCRJUtuwq0iSpBpycK7Uxjo6Ojj9+D249rzPcNXZn+KtKy/9mu2f2ff9APzhh0ew/f9bB4ARw4Zyzsl7cfUPj+C6nxzF+LVXbXm7pQXR1dXF4YcezFZbbMq279uahyZOfM32c885m/Hjx7Pl5ptw+e8ue822G66/jnGrr9zK5kr9YuGiRcLO71mPYUMHs/U+p/H50y/hlCM/NGfb2uNW4MPbjwfgA4ecwecP2ZHhw4bw6X3ez30Tn+T9n/gWh578v7xttWWqar70pvz2kt8wY8YM/nzDXzj5y6dwzGc/M2fbpEmTOPOM07nxxhu59PKr+MLxxzJz5kwAHnvsMb79zdOY9corVTVdA6Cjo3W3VrJw0SJhsw3X4A833Q/Arfc8wrvescqcbWutvizX3/53AGa+PIuHHn2add+2Itts9nZefmUWv/3uYRx7wHZcXT5eahc33XgD2/zHdgC8e5NN+L//u33Otttvu5VNN9ucxRZbjDFjxvDWceO45+67mTFjBp887GC+9Z0zq2q21KemFi4RsXVE/LIJx105Ii6IiGsj4paIODMihg7087xZEbFKROxUdTv0qsVHDmPKtJfmLM+e3cWgQcXH/96JT7D5O8cBsOSYkWyy/lsZMXwoS40dydjRI9j5sO/yu+vu5atH7lJJ26U364WpUxkzZsyc5UGDBjFr1qw520Y3bFt81OJMnTqFT39yAkcceRQrrrhiy9urgdXR0dGyWyu1XeISEYOAS4DTMnPrzHw38ApwUrUte433AptX3Qi96oXpM1h8xGJzljs7O5g9uwuA/MdTfP/86wA45chduO3eR5j8/HSefX46v/vzPQBc/ud7eGdDSiO1g8VHj+aFF16Ys9zV1cXgwYPnbJvWsO2FaS8wdOhQbrzxer588ols+76tefbZZ9lrzz1a3m6pL5WcVRQR2wBfAmYAk4GPAz8GvpSZt0dEAsdk5sUR8Xtgv8x8vHz4FsBjmXlLwyGPpizCIuIzwB7ALOC6zDw6Ik4AxgFLA0sCZwK7AmsC+wCTgPOBx4DVgF8C6wAbAr/LzOMiYl3gdKCjoc0bls/9MrB6eYxTgGOAERFxE7BS+RxdwA2Z+d8D8BbqDfrLnQ+zw5br8Ks//JWN112Neyc+MWfb0kuMYumxIwE46tSLuOzMCfxt4hPcdOfD/McWa/PX+x9ji3eN4/6Hnqyq+dKbsulmm3P5ZZfyX7t9mFtuvpl11ll3zrbxG23MCZ8/nhkzZjBl+kzygfsZv9HG3P23nLPPaistx09/PuChuVqkpicV0dHd3d20g0fE1sDBmblHw7oO4GFgi8x8PCI+BawM3A2sCvwM+A1wI3AscGlmbtnw+I8AG2fmp+fxfOsC51AUN7OAXwHnAuOBFTPzgIg4BnhnZn44IvYD1ge+BdxGUdwMB/4BrAi8CPwzM5eNiJuBj2fmfRHxCeCtwB8oiqD1gMWAJzJzTETsC6yVmcdExG3A4Zl5c0QcApydmbN6e8/+NvGJ7rXHrdDv91iSetPV1cWhhx7K3XffTXd3Nz/60Y+4/PLLGTduHDvvvDNnn302Z511Fl1dXRx33HHsuuuur3n8csstx6RJkypqff3NmAXDBtO08mL8l/7UvC/4udz+ufe0rEyqInFZGpjakKBcB3wF+DJFF9AzwNeAI4HtgUvnevw/KdKSOSJiKWBTiqLj5sx8pVx/PbB2udsd5c/ngfvK+88Bw8r7D2fmlIiYCTyVmc+Wx+j5xb8dODMiAIYAD5br7ykLkVkR8eogilftBxwVEV8D/gJ9f0jH7/aVvjariV766xkM33BC1c1YJD132xlVN6GmOvnWGd9/zZpDP7kWUHxp7rXfARxwwAHMKP8pNWOuf1I98q9Jr1un9uE8LgPnGWB0RCxfLm8FPJiZz1EkHLsDVwKPAkcAv57r8TcDq0fExjAnwTkB2BJ4AHh3RAwu12/JqwXG/CrP+W1PYO/M3Br4LPC7Ph7Xxavv7QEUqdNWFF1Lm83neSRJUi9akbhsGxG3Nyx/lOLL/NcR0UWReuxbbruEYjzLsxFxFXBoZj7UeLDM7IqI3YAzImIkMJKimPlcZr4cERdQdDN1AjdQdDutPwCv4xDgJ+XgYIBPAL316dwDHB8Rd5T3b4uIfwOPA7f08hhJkgZMTQOX5o5x0Rs3fMMJ/kIqYldRdewqqs6wwa/vIlLrNHOMy8ZfubZl3ye3Hrd1rce4SJKkJnOMiyRJUsVMXCRJqqGaBi4mLpIkqX2YuEiSVEOOcZEkSaqYiYskSTVU08DFxEWSJLUPExdJkmrIMS6SJEkVM3GRJKmGahq4mLhIkqT2YeIiSVINOcZFkiSpYiYukiTVkImLJElSxUxcJEmqoZoGLiYukiSpfVi4SJKktmFXkSRJNeTgXEmSpIqZuEiSVEM1DVxMXCRJUvswcZEkqYYc4yJJklQxExdJkmqopoGLiYskSWofJi6SJNVQZ00jFxMXSZLUNkxcJEmqoZoGLhYukiSpeSKiEzgTWB+YCeyfmRMbtm8PfLFcvAM4LDO7ezueXUWSJNVQR0dHy27z8UFgWGZuChwDnNazISIWB04FPpCZmwCPAEv3dTALF0mS1ExbAFcCZObNwPiGbZsB9wCnRcT1wFOZ+e++DmbhIklSDXV2tO42H6OBKQ3LsyOiZ6jK0sB7gKOB7YEjImLNPl/Xm3s7JEmS+mUqsHjDcmdmzirvTwZuy8xJmTkNuA7YoK+DOThXkqQaWoiuVXQjsBNwQURsQtE11OP/gHUiYmngeWAT4Oy+DmbhIkmSmuliYJuIuAnoAPaLiCOBiZn524g4Friq3PeCzLy3r4NZuEiSVEMLS+CSmV3AwXOtfqBh+y+BX/b3eI5xkSRJbcPERZKkGupgIYlcBpiJiyRJahsmLpIk1VA/5ldpSyYukiSpbZi4SJJUQwvRPC4DysRFkiS1DRMXSZJqqKaBi4mLJElqHyYukiTVUGdNIxcTF0mS1DYsXCRJUtuwq0iSpBqqaU+RiYskSWofJi6SJNWQE9BJkiRVzMRFkqQaqmngYuIiSZLah4mLJEk15AR0kiRJFTNxkSSphuqZt5i4SJKkNmLiIklSDTmPiyRJUsVMXCRJqqHOegYuJi6SJKl9mLhIklRDjnGRJEmqmImLJEk1VNPApffCJSK+0NcDM/OkgW+OJElS7/pKXGpaq0mSVH91HePSa+GSmSf23I+IkcAawL3A8Myc3oK2SZIkvcZ8B+dGxHuBu4BLgGWAf0bEts1umCRJevM6O1p3a+nr6sc+XwW2AJ7PzEnAlsCpTW2VJEnSPPSncOksCxYAMvO+JrZHkiQNgI6OjpbdWqk/p0P/KyI+AHRHxFjgMODR5jZLkiTp9fqTuBwE7AmsDDwMbAAc2MxGSZKkBdPRwlsrzTdxycyngY9ExGhgVma+2PxmSZIkvd58C5eIWBc4D1gF6IiI+4F9MvOhZjdOkiS9OZ01ncelP11F3weOz8ylM3Mp4DTg3OY2S5Ik6fX6U7gMz8wrehYy82JgdPOaJEmSNG99XatolfLuXRFxDPBDYBbFQN3rW9A2SZL0JtW0p6jPMS5/BropBgxvTXF2UY9u4JPNa5YkSdLr9XWtotVb2RBJkjRwFrmLLPaIiLcBE4BRFOnLIGD1zNyyyW2TJEl6jf4Mzv0F8DywIXAnxWnR9zazUZIkacF0dLTu1kr9KVyGZuYXgSuBO4AdgK2a2ipJkqR56E/h8mJELAY8CLwrM19qcpskSdIC6uzoaNmtlfpzkcWfAZdSnAb9l4jYDni8qa2SJEmah/kmLpl5BrBrZv6b4rTos4APNrldkiRpAdR1jEtfE9B9Ya7lxsV1gZOa1CZJkqR56qurqJ4ngEuStAhY5OZxycwTW9kQFX5w9tFVN2GR5vtfje3OuLHqJiyyrj1ic9//Cl17xOZVN6Ht9GdwriRJajP9OW24HdX1dUmSpBrqV+ISESOBNYB7gBGZOb2prZIkSQukrmNc5pu4RMT7gLuAS4BlgX9GxLbNbpgkSdLc+tNV9BVgC+D5zJwEbAmc2tRWSZKkBdLZ0bpbS19Xf/YpCxYAMvO+JrZHkiSpV/0Z4/KviPgA0B0RY4HDgEeb2yxJkrQgWp2EtEp/EpeDKK5TtDLwMLABcGAzGyVJkjQv801cMvNp4CMtaIskSRogdT2raL6FS0T8A+iee31mvrUpLZIkSepFf8a4bN1wfwiwC7BYU1ojSZIGRF3HuPSnq+ifc606NSJuB77UnCZJkiTNW3+6irZsWOwA1gaGN61FkiRpgdV0iEu/uooarxLdDTwD7NOc5kiSJPWuP4XL+Zn5/aa3RJIkDZjOmkYu/ZnHZULTWyFJktQP/UlcHouIa4BbgJd6VmbmSU1rlSRJWiD9SSbaUX8Kl5sb7tczd5IkSW2h18IlIvbJzPMy88Te9pEkSWqlvpKkT7WsFZIkaUB1dLTu1kp17QKTJEk11NcYl7Uj4uF5rO8Aur1WkSRJC6+6ng7dV+EyEdihVQ2RJEman74Kl5fncZ0iSZLUBmoauPQ5xuXGlrVCkiSpH3pNXDLTGXMlSWpTnYtg4iJJkrRQ6c/MuZIkqc3U9awiExdJktQ2TFwkSaqhmgYuJi6SJKl9mLhIklRDnlUkSZJUMRMXSZJqqIN6Ri4mLpIkqW2YuEiSVEOOcZEkSaqYiYskSTVk4iJJklQxExdJkmqoo6ZT55q4SJKktmHiIklSDTnGRZIkqWImLpIk1VBNh7iYuEiSpPZh4iJJUg111jRyMXGRJEltw8RFkqQa8qwiSZKkilm4SJKktmFXkSRJNVTTsbkmLpIkqX2YuEiSVEOd1DNyMXGRJEltw8RFkqQaWljGuEREJ3AmsD4wE9g/MyfOY5/fAZdk5vf7Op6JiyRJaqYPAsMyc1PgGOC0eezzJWDJ/hzMwkWSpBrq7GjdbT62AK4EyMybgfGNGyPiv4Au4Ip+va43/E5IkiT132hgSsPy7IgYDBAR6wAfBb7Q34M5xkWSpBpaiC6yOBVYvGG5MzNnlff3BlYErgFWA16OiEcy88reDmbhIkmSmulGYCfggojYBLinZ0NmfrbnfkScAEzqq2gBCxdJkmpp4QlcuBjYJiJuAjqA/SLiSGBiZv72jR7MwkWSJDVNZnYBB8+1+oF57HdCf45n4SJJUg0tRGNcBpRnFUmSpLZh4iJJUg3VNHAxcZEkSe3DxEWSpBqqazJR19clSZJqyMRFkqQa6qjpIBcTF0mS1DZMXCRJqqF65i0mLpIkqY2YuEiSVEPOnCtJklQxExdJkmqonnmLiYskSWojJi6SJNVQTYe4mLhIkqT2YeIiSVINOXOuJElSxSxcJElS27CrSJKkGqprMlHX1yVJkmrIxEWSpBpycK4kSVLFTFwkSaqheuYtJi6SJKmNmLhIklRDdR3jYuGiRUJXVxc/+drxPPr3+xkyZCgf/9z/sOzKq83ZfvUF5/GdQy5h8ouv8MH9P8UG/+/9vDhtKmceN4GZM15k8OAhHHTStxm79DLVvQjpDeoAPv3eNVjjLSN4ZXY3p/5hIo9PmTFn+8arjQXgzN3X48Gnp/GtPz3MyKGDOH67NRkxdBBDBnXw3ese4b4nX6joFUivZ1eRFgl3XHsVr8ycyRfO/Q27TTiGX3zr5DnbXnj+Wf540U+46aabOPrMX3DeKcfT3d3NDZdeyMrjguPPuoh3b7MTV/z0BxW+AumN22KNJRk6uIPDzr+Hs254hEO2XG3OtuFDBnHIFsXyoeffzaSpMxkzfDC7vXMF7njseY646F5O+f1EjnjPW6tpvBZYZwtvrWThokXCg3fdxrqbbQ3AuHXfyT/uv3vOtsXHLsmX/vcqhgwZwpTJ/2bE4qPp6OhgpXFr8dKL0wF4afo0Bg02oFR7WXfF0dz6yPMA3DdpGrHsqDnb1llhcR6e/CIAp++2Ds+9+DJTXprFRX99gt/e/RQAgzrg5VldrW+41Iem/CWOiK2BgzNzj4Z1pwAPAHcCO2fmSb08dl9grcw8po/jLwF8HXgbMAh4DDgoM6cM1GtYEBGxJLBdZv5v1W1R4aXp0xg+cvE5y52dg5g9a9acYmTQ4MGcccYZnPS5z7PN7vsBMGrMEtx783Uc++H3Mn3KFI47+6JK2i69WSOHDmbay7PmLHd1FcXI7G4YM2wIG640BoCjf3Mfp++2Ln978gX+9XzRlbTkiCEcv92anPHnf1TSdi24uo5xaXnikpl39la0vAG/AC7LzC0zc3PgFmBhyvHXA3auuhF61fCRo5jx4rQ5y93dXa9LUCZMmMDpV9zOg3fcwv2338Rvzv4WO+59MF+94BqOOuNnfOfog1rdbGmBTH95FiOGDJqz3FkWLQBTZ7zCA08V/0+89EoXdz0+lXFvGQnA6kuN4LRd1+Hsm/7JXY9PbXm7pb60PPtuTGMi4hPABOBZ4GXg/HK3TSLi98BbgO9l5lkNj18VWC4zL2447OnAqHL7nsARwEzg78CBwJ7ATsBwYHng28B/AusAR2XmJRExEbiJIsW5BhgDbAxkZu4VESsDZwHDgBnlcQdRFFGPAWsAt2bmIcDxwPoRcSDwDHA08ArwCLB3Zpq9ttjb1h/Pndddzbu32YmJ99zBSmusNWfbk488xIXf/Rp7XXM5gwYPYfDQxejo6GTk6DEMHzUagNFLLMWM6dN6O7y0ULr3iRfY7K1LcO3fJ/OO5UbN6RoCyKensfpSI4AihXnH8ovzu3ufYtUlh3PijsGJlycPPfNib4dWG6hn3tLcwuW9EXFtw/JbgS/0LETE0hRf6BtQFBl/atj3FeA/gFWByykKhh4rAK/JLjNzNjAlIpYCTgQ2zMwXIuKbwEHANGDxzNw2IvYAPg1sAmwNfAq4BFgNeC/wJEUh9W7gcODhiBhL0TV1emZeERHvA06hKFDWBLYFXiz3XQ74MkVxdlZEXAh8MzN/GRF7A6OB59/A+6gB8K6tt+Nvt1zPyR/fhW662f8LX+fKn5/NMiutyju32paV13w7m266Kc+8+Arrbbo1a71rE5ZdZTXO/dLR/PGinzB71iz2O/6Uql+G9IZcP3Ey41cZyxkfXpeODvgPQD0EAAAV9klEQVTa7yey24Yr8PiUGdz08LOcfeM/+cIOwZl7rM+1f3+Gf0x+kS/ttBZDB3dy+NbFoNxpM2fxuUsfqPiVSK9qZuFyzTzGuDQaB9yXmS+W229q2HZHZnZHxCRgxFyPexRYqXFFRAwBdqNIWP6WmT3n7l1HUVTcAvy1XPc8cH95/OcoEhSAyZn5aHm86Zl5X3l/SrnPusBxEXE0RSH7cvm4iT3PFxFPNhyvx5HAsRFxCHA/8JvXv1Wv2mnt5Vhi+JC+dtGbtO+vfvbaFf/1njl39x7/jXk8YmU+tf01zW2UANh7/MpVN2GR8JN93jnP9bHsKGLZURxUnmUEsNzoV/+UXXvE5s1u2iJp62/d2NTj13SIS6XzuEwE1oqI4RSJy8YUg3cBunt7UGY+HhHPRMR/ZuYl5epPlY8/FHhHRIzMzOnAVsCD8ztmP7c/AHw9M2+KiLXKY/f2uC5eHT90IHBCZj4dET8AdgHO6+1JLv3bpPk0Q82y9/iV+cntj1XdjEXSuTc8WnUTFlnXHrF5079ApYFUWeGSmc9ExNeA6ym6ZoZTdBH1J27YC/huRBwFDAUeAg7IzCkR8UXgTxHRRVEcHQPs0fuh+u0o4HsRMaxs66f62PchYN2IOAK4FfhDREwGXgAuG4C2SJLUp86ajnLp6O6eX9DQHBExGDg6M79cLl8HfC4zr6ukQQuJn9z+WDW/EJm4VMjEpTomLtW69ojNm1ZdXHrPUy37Ptlp3WVbViVVmbjMioiREXEHxXiRWyjSF0mStIAc49IEmXkccFyVbZAkSe3DOcwlSaqhjpqOcfFaRZIkqW2YuEiSVEN1HeNi4iJJktqGiYskSTVU13lcTFwkSVLbMHGRJKmGHOMiSZJUMRMXSZJqyMRFkiSpYhYukiSpbdhVJElSDTnlvyRJUsVMXCRJqqHOegYuJi6SJKl9mLhIklRDjnGRJEmqmImLJEk15AR0kiRJFTNxkSSphhzjIkmSVDETF0mSash5XCRJkipm4iJJUg05xkWSJKliJi6SJNWQ87hIkiRVzMRFkqQaqmngYuIiSZLah4mLJEk11FnTQS4mLpIkqW2YuEiSVEP1zFtMXCRJUhsxcZEkqY5qGrmYuEiSpLZh4iJJUg15rSJJkqSKmbhIklRDNZ3GxcRFkiS1DxMXSZJqqKaBi4mLJElqHxYukiSpbdhVJElSHdW0r8jERZIktQ0TF0mSasgJ6CRJkipm4iJJUg05AZ0kSVLFTFwkSaqhmgYuJi6SJKl9mLhIklRHNY1cTFwkSVLbMHGRJKmGnMdFkiSpYiYukiTVkPO4SJIkVczERZKkGqpp4GLiIkmS2oeJiyRJdVTTyMXERZIktQ0TF0mSash5XCRJkipm4iJJUg05j4skSVLFTFwkSaqhmgYuJi6SJKl9mLhIklRHNY1cLFwkSVLTREQncCawPjAT2D8zJzZs/zSwR7l4eWae2Nfx7CqSJKmGOlr433x8EBiWmZsCxwCn9WyIiLcCewKbAZsC20bEen0dzMJFkiQ10xbAlQCZeTMwvmHbY8B2mTk7M7uAIcCMvg5mV5EkSTW0EM3jMhqY0rA8OyIGZ+aszHwFeCYiOoBTgb9m5oN9HczERZIkNdNUYPGG5c7MnNWzEBHDgJ+X+xw6v4NZuEiSpGa6EdgBICI2Ae7p2VAmLZcAd2XmQZk5e34Hs6tIkqQaWnh6irgY2CYibqJo1n4RcSQwERgEbAUsFhHbl/sfm5l/6e1gFi6SJKlpykG3B8+1+oGG+8PeyPEsXCRJqqOFKHIZSI5xkSRJbcPERZKkGurHxHBtycRFkiS1DRMXSZJqaCGagG5AmbhIkqS2YeIiSVIN1TRwMXGRJEntw8RFkqQ6qmnkYuIiSZLahomLJEk15DwukiRJFTNxkSSphpzHRZIkqWId3d3dVbdBkiQNsIeefqllX/BrLDO8ZfmOiYskSWobjnGRJKmOHOMiSZJULRMXSZJqyHlcJEmSKmbiIklSDTmPiyRJUsUsXKQmiIjO8mdN/80jzVtEDKq6DSp0tPDWShYu0gCLiM7M7IqI5YBPRcTyVbdJaoWIGJSZsyOiMyIOjoglqm6T6seZc6UmKIuV3wA/BmYClwLTM/PFKtslNVuZNv4KSOCvwBUAmTm1ynYtih6ZPKNlX/CrLTXMmXOldhMRG0ZEz4D33YDLKIqXXYEzgO2qapvUTBHx9obF3YGpwEnAh4GTgV2qaJfqycRFGgARMQzYERgP3AOsALwLmA4cD+wDLA0cnZn+T6faKMe0fBjYELgTGAtsBMwCzgfGAesBh/nZb61/Tp7Zsvd71aUWM3GR2klmzgC6gM8C22bm1zPzI8CtwObAfwFn+YdbdZOZs4H7gX2B3TPzTGB/ii6i54GPAhf62ddAsXCRFsBcZ1D8FjgEeCQijinXvQSsCeyXmRNb3T6pWeb67N8PnAo8FBFfKYuZscAOwNcz809VtFH1ZFeR9CY1nD20IvB5isG3n4mIDYA9gVWAa4EfZ+ZLFTZVGlCNZw8BpwBPAN8DVgUmAKOBu4FzMnNqRHSYuLTeo8+2rqtolSVb11Vk4SItgIhYGjgLuJpijEtnZm4fEasCBwA/ycwHq2yj1AzlHEXfA14E1gceBI6jSFo+A1ycmX+sroWycJH0GhGxGPAVYPXM/FC57nxglczctNLGSU0WEf8DrJ2ZO5bLPwdmAIf3nPZv0lKtx1pYuKzcwsLFMS7SG9AwI+6gzJwJ/BlYIiIOAsjM3YEsExepNuYxI+7vgXUjYu9yeW+KtOWtPTtYtKgZTFykfmoY07IC8DXgRmAIcB/F6aD3ZuZ3qmyj1AwNn/1O4BvAA8ALFJPMfQX4ZWaeW2Ub9Xr/eq51ictKS9hVJC2UIuItwK+B71LM1bIN8EOKiHxnitOhp/gvTdVNOablfOA24BHgc8DRFJ/9U4APAU+VZxRpIVDXwmXw/HeRFm1z9dMvS3Ha8zXAhRQFzCzgeuBPmTm9mlZKA68naelZBB6jKNTPBc4E3kIxvf+OmTm5mlaqd/W8xquJi9SHhoh8BDAKGAbcAEymmMp/TeBYYCevxaI66Tnluby/OMVn/6cU3aOnU8zdcg6wR2Y+UVlD1at/PfdyCxOXoQ7OlapWJi0987RcTHGhxGUoZgVdFngvcAJwiEWL6qT87Pdc5fnnFNfcWpIibVwKGEpRtHzNomXh1dHRultLX5eJi/R6DRNsLUZxsbhbgGnAecC2wBRgJWBSZj5cXUulgTVX0nIsRVfos8CBwF4UEysOAl7KzOsqa6jm6/HnW5e4rDi2dYmLY1ykuTQULSsC/01x8bgzMvOxiDgUuJ2iT//qShsqDbC5ZsQ9BtgUODgzH4+IoRTjWQ7MzL9U2lD1Sz1HuNhVJL1O+Yf7LcBXKaYtvwO4PCJWyMyLgV2AR6tso9QM5We/g+KzP4QiZfx1ue17FIPR/d5Qpewqkko9Zw+Vf7g/D+yVmW8rt50AfALYODOfrLCZ0oBrPHsoIk4C9snMVcvlnwLrZOaGVbZRb9yTU1rXVbT8GAfnSi1V/uHujojRFLN//hC4OyJ+BJCZJwDfB4ZX10pp4JXdQz1Fy1DgZODZiDgXIDP3Ah6MCC9joYWCiYsWeQ1Jy/LAVRRjWB4CfkIxodxymblblW2UmqHhs98JXA48RzHB3Lcp/l94KjP3rLKNevMmTXmlZV/wy40ZYuIitUJD0jKc4myhb1JMab4UsCdwGvBYWdRItVEmLT1fbHsAN1N0kW4EfAr4D2C1iFi35xpd0sLAs4q0yGqYXG55iplAXwAuycx7I+IXFPO17Ap8NjNnVdlWaSCVn/2es4fOoRiI+8fMnBgRnwe+DgzJzM0rbagWTE1PK7KrSIu0iBhLkbDcDrwMrANcnpm/j4gNgScy86kq2yg1QzkI/dvAdIqLJu4I/DQzL42IccCozLyzyjZqwUya2sKuotGt6yoycdEiZ65+/QkUF0u8EngS+AiwW0R0OU+L6mau627tBWwM7J2ZD0bEdGDfiBiSmb+urpUaKDUNXBzjokVLT79+RIykiMf/RDFPy4eApYELKC6YeE91rZQGXuOYljJtSeAy4OCIWCMzL6L4/D9UYTOl+bKrSIuMhqRlJYqLxEFxwcShwGLl8tnAkw3/KpXa3lwp41kUafs/KMZ1TQfeDpzu5Svq5ekXWtdVtMzinlUkDbiGeVrOBc4Hjis3dQN/obj+ykyLFtVN+dkfDFxIMevzd8qfqwCPU5wGPaS6Fkr95xgX1VoZiW9GUZT8DRgFzMrM88vts4BvAWcA13j2kOokItah+Oz/naIw6crMk8ptk4H3UBTtf8jMmZU1VE3RUdNRLiYuqq0yFr8Q+DTwMWCxzPwn8HB5yifAqhTdRMMtWlQnEfEN4GsUkyiunZlTgM6I+Hq5y2BgDWBpixa1E8e4qJbKpOU8YGJmnhQRozJzWkQsDmxFkcK8k2J8yycz894KmysNqIg4BVgSOBgYkZnTyvXrUkwutyLFJIsnZeZllTVUTfXvabNa9gX/llGDPR1aWkBjgMk9sTjwUvlzWYpTQL9GcRr005k5uYL2SU1RjuNaAvhMOcHii+X6t1HMBj0BWAuY6mBctSMLF9VO2UX0FuD9EbFyZj7Gq1MaLA+sUsbmU6pqo9QMZdI4gmLQ7dBydSfQBcwCVsvMGYATyy0C6jnCxTEuqqHM7MrMvwMXA4dFxBIN41dWAEZExIjyj7xUG5nZnZmTgIeBU8u5W3o++ysBS0bEkl57SO3MMS6qnYY5KzYCdqLoz/8BxVwV+wKHZeZ9FTZRaoqG62+tQDEod1ngi8BqFBdQ/FJmXlVhE9VCk6e3bozLUiNbN8bFwkW1EhGDG88OioiVgd0oipZu4BuZ+UBV7ZMGWpmqzJ7H+lHAF4CxwEjgR17GYtFi4SIt5CJiOMXEWqdk5sSq2yO1SkQMAlbv+dzPXcyU1x96pbIGqhJ1LVzs51Rb6+mrL5OWl4C7gBMiYkS53nEsqqWI+GhEbFcu/go4LSIuA8jM2RExtGF35yhaBHW08L9WsnBRWyv788cAv4qIw4DLgT8CK5e7WLiodiJiLMWZc9tFxIXArZn5n8CUiLgUIDNf7tnfy1ioTixc1JYaz4ooT21+GdgP+DnFXBX7ltu6qmif1CxlN9DzFLNC300xZ9EkgMzcE3glIv5QYRO1kOjoaN2tpa/LMS5qVxGxBHAIRcIyA1iOYgDuxyimMt8JeM5/baouesaulIX7HhRXdl4GWBf4fc8suBGxQWY6V8si7rkXZ7fsb98SIwY5xkWal4YxLcMoUpalKWbC/Q5wEPByZu4N7JCZz1q0qC7KU517ipYfAMcCWwDPAHcA20fEBwEsWlRnFi5qGw1zVCxPEZMfTXG150soCpflgS9GxIiy+0iqhXIG6J5uzx9TTDC3M0XxviEwGfgrcH8lDZRayMJFbaMsWsYCv6YoVi4C/gF8LjMvBPYGdsvMFytspjSgyokUt2xYNRL4VWb+A/g2sDXFBUNvysz0TDr1qOsYF69VpHazJHB/Zp4DEBEzgU8DlNP8S7UREe+hGKf184j4LHAFcCXwlYg4nOJv+CvAe4F/AffZPaq6M3FRu5kCbBURB5bL6wFrlgN1pdqIiDWBvYAtImJDYCZwDPAEcA3wS4qz6D4M/AhYKyIGmbioR13ncfGsIi20IuIE4ObMvLJcHpyZsyJiPHAB8HtgI2BPp/FXHUXEFsB/AE8DNwLrANsAXwUeopgC4N8UU/t/ODOzoqZqITTlpa6WfcGPGd7pWUVatEXEYhRzU+xdxuUAXeXpoLdT9Ot/E9jRokV1EhFr9dzPzBuAEcD+wKbAg8ANwAlAF5AUZ9XtYdGiudV1jIuJixZa5bWH/gv4AHBWZv6xXH8AsAJwshPMqU4iYjPgRODLmXltROwL7AL8jGKulmeBm4F/Zea/ysd4HSLN09QZrUtcRg9rXeJi4aKFWlm87AbsCJwMrETxr80DMvOeCpsmDbjyGlu7Au8DngNWAyZk5uMRsQ3F/wffyMxHI6LDgbjqywstLFwWt3CRXlUWLx8CPktxKugH7B5SXZWf9z0oxq0cm5m/bJjDaInMfK7iJqpNWLhIFWpIXm61aFHdlWO89qAYy3V+zwB16Y14YWYLC5fFLFyk1zEa16KkLNY/CmwPHJiZz1bcJLUZCxdJUkuVxcuSmfl41W1R+5k2s3Vf8KMWa925RRYukiTVUF0LF6f8lySphlo9v0qrOAGdJElqGyYukiTVUE0DFxMXSZLUPkxcpEVQRKxGcd2b+4BuYCjFVYf365lK/k0cc19g68zcNyIuB/bPzCd62fdE4OrMvP4NHL87MzvmWncCQGae0MfjHinb9Ug/n2e+x5TaQk0jFwsXadH1RGZu0LMQEacBpwIfWdADZ+YO89llK+BPC/o8khY9Fi6SevwJ+CrMSSluATYA/h+wHXAERffy/wGHZeaMiNgL+BwwFfgnMK3h8VtTXOH7u8AWwCsU15taDBgPnBMRuwAvAd8DlgJeBA7PzL+WqdDPgFEUFxbsU0RMAPaiuCzEy8BHGq6YfEJErA/MAA7KzLsjYlngB8DKFFdaPjYzr35D75i0EOuoaeTiGBdJRMQQiitx/6Vh9RWZGcBbgAOAzcqE5mngqIhYAfgfYEtgU2DxeRz6cIrC4+3A+ymuv/NL4HaKrqR7gPOAz2bmO4EDy+0AZwA/Lp/zxvm0fzTwQYouoXWAy4AJDbv8PTM3pCiczivXfRs4NzPfBewM/CAi5vUaJC1ETFykRdcKEXFneX8x4FbgmIbtt5Q/3wO8Dbg5IqAYD3MHsBlwU2Y+BRARP6O4qnGjrYCzMrOLIn1Zu9yX8ucoYCPgRz3rgFERsRRFYtPTbfVz4Ie9vZDMnBoRHwX2iIg1KRKiOxt2Oafc7/KI+FlEjKUopNaKiJPKfYYAa/T2HFK7qes8LhYu0qLrNWNc5uGl8ucg4ILM/CTMKTYGUxQpjX8aZ83jGK9QDP6lfOw44NGG7YOAGXONtVkJeLZ8XE8q3A3M7q2hEbEycC1FSnMFRZG0YS9t6yjbNQh4b881gCJieYo06YO9PY+k6tlVJGl+rgV2iYhlIqKDYjzKEcANwKYRsWJEdAK7z+Ox1wG7R0RHRCwD/Jki3ZkFDM7MKcDfI+JjABGxTfkYgKuBj5X3PwQM66ONGwETM/ObwG3ALhSFSY89y+PvAtyfmdOBa4BDy/XvAO4FRvTvLZEWfsMG09GqWytfl4WLpD5l5l3AiRRf9H+jKAhOKbuIDqcoMG6lGKA7tzOB6cBd5X6HZ+YLwJXA9yNiM4qiYv+IuJticPDu5VXAJwC7RsRdwA7AC3008/dAZ0TcR9GN9QCwesP2NctusSOBfcp1hwOblM97PvCxsm2SFmJeZFGSJLUNExdJktQ2LFwkSVLbsHCRJEltw8JFkiS1DQsXSZLUNixcJElS27BwkSRJbcPCRZIktY3/D3RCIUmNfE5yAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 576x576 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import itertools\n",
    "\n",
    "# From ScikitLearn's documentation:\n",
    "def plot_confusion_matrix(cm, classes,\n",
    "                          normalize=False,\n",
    "                          title='Confusion matrix',\n",
    "                          cmap=plt.cm.Blues):\n",
    "    \"\"\"\n",
    "    This function prints and plots the confusion matrix.\n",
    "    Normalization can be applied by setting `normalize=True`.\n",
    "    \"\"\"\n",
    "    if normalize:\n",
    "        cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]\n",
    "        print(\"Normalized confusion matrix\")\n",
    "    else:\n",
    "        print('Confusion matrix, without normalization')\n",
    "\n",
    "    print(cm)\n",
    "\n",
    "    plt.imshow(cm, interpolation='nearest', cmap=cmap)\n",
    "    plt.title(title)\n",
    "    plt.colorbar()\n",
    "    tick_marks = np.arange(len(classes))\n",
    "    plt.xticks(tick_marks, classes, rotation=45)\n",
    "    plt.yticks(tick_marks, classes)\n",
    "\n",
    "    fmt = '.2f' if normalize else 'd'\n",
    "    thresh = cm.max() / 2.\n",
    "    for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):\n",
    "        plt.text(j, i, format(cm[i, j], fmt),\n",
    "                 horizontalalignment=\"center\",\n",
    "                 color=\"white\" if cm[i, j] > thresh else \"black\")\n",
    "\n",
    "    plt.tight_layout()\n",
    "    plt.ylabel('True label')\n",
    "    plt.xlabel('Predicted label')\n",
    "\n",
    "cnf_matrix = confusion_matrix(y_test, predictions)\n",
    "np.set_printoptions(precision=2)\n",
    "\n",
    "plt.figure(figsize=(8,8))\n",
    "plot_confusion_matrix(cnf_matrix, classes=['Low Comments','High Comments'],\n",
    "                      title='Confusion matrix')\n",
    "\n",
    "plt.figure(figsize=(8,8))\n",
    "plot_confusion_matrix(cnf_matrix, classes=['Low Comments','High Comments'], normalize=True,\n",
    "                      title='Normalized confusion matrix')\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Step 6: Answer the Question\n",
    "#### Interpreting the Feature Importances\n",
    "The models I chose are extremely difficult to interpret. Seeing as how the model only improved by 0.1 from the baseline, I didn't expect to see any very high coefficients. The highest from the RandomForest model are in the thousandth decimal range or lower, and they are all dummies of subreddits. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {},
   "outputs": [],
   "source": [
    "rf_feats = pd.DataFrame(\n",
    "    list(\n",
    "        zip(X_test.columns, np.abs(\n",
    "            rf_gs.best_estimator_.steps[1][1].feature_importances_))), columns=['feature','coef_abs'])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>feature</th>\n",
       "      <th>coef_abs</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>536</th>\n",
       "      <td>Lovecraft</td>\n",
       "      <td>0.001057</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>196</th>\n",
       "      <td>Catloaf</td>\n",
       "      <td>0.001040</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1715</th>\n",
       "      <td>robotics</td>\n",
       "      <td>0.001030</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>891</th>\n",
       "      <td>Torontobluejays</td>\n",
       "      <td>0.001023</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1177</th>\n",
       "      <td>crossdressing</td>\n",
       "      <td>0.001020</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>586</th>\n",
       "      <td>MilitaryPorn</td>\n",
       "      <td>0.001007</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>661</th>\n",
       "      <td>OutOfTheLoop</td>\n",
       "      <td>0.001002</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>156</th>\n",
       "      <td>Braves</td>\n",
       "      <td>0.000967</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1252</th>\n",
       "      <td>entwives</td>\n",
       "      <td>0.000939</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>187</th>\n",
       "      <td>CanadianForces</td>\n",
       "      <td>0.000917</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              feature  coef_abs\n",
       "536         Lovecraft  0.001057\n",
       "196           Catloaf  0.001040\n",
       "1715         robotics  0.001030\n",
       "891   Torontobluejays  0.001023\n",
       "1177    crossdressing  0.001020\n",
       "586      MilitaryPorn  0.001007\n",
       "661      OutOfTheLoop  0.001002\n",
       "156            Braves  0.000967\n",
       "1252         entwives  0.000939\n",
       "187    CanadianForces  0.000917"
      ]
     },
     "execution_count": 100,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rf_feats = rf_coefs.sort_values('coef_abs', ascending=False)\n",
    "rf_feats.head(10)\n",
    "# These are all subreddit dummies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 156,
   "metadata": {},
   "outputs": [],
   "source": [
    "lr_coef_list = gs.best_estimator_.steps[1][1].coef_[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 157,
   "metadata": {},
   "outputs": [],
   "source": [
    "lr_coefs = pd.DataFrame(\n",
    "    list(\n",
    "        zip(X_test.columns, np.abs(\n",
    "            lr_coef_list))), columns=['feature','coef_abs'])\n",
    "lr_coefs.sort_values('coef_abs', ascending=False, inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 158,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>feature</th>\n",
       "      <th>coef_abs</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "Empty DataFrame\n",
       "Columns: [feature, coef_abs]\n",
       "Index: []"
      ]
     },
     "execution_count": 158,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lr_coefs[abs(lr_coefs['coef_abs']) > 0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Conclusions\n",
    "Though the models have around 90% accuracy, I was hoping for a greater improvement from the baseline. The number of comments on a reddit post inevitably comes down to human activity, which is inherently unpredictable. Were I to create a classification model for this problem again, I would have created more than one class for number of comments. Based on the extremely skewed distribution of number of comments, it would have been prudent to create a \"low comment\" class, a \"high comment\" class, and a \"ultra-high comment\" class. Likewise, although the purpose of this project was to use a classification model, a regression analysis would have made sense as well. "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [conda env:dsi]",
   "language": "python",
   "name": "conda-env-dsi-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}