Data Science with Python- Pandas .ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Pandas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[   0,    1,    2, ...,   47,   48,   49],\n",
       "       [  50,   51,   52, ...,   97,   98,   99],\n",
       "       [ 100,  101,  102, ...,  147,  148,  149],\n",
       "       ...,\n",
       "       [4850, 4851, 4852, ..., 4897, 4898, 4899],\n",
       "       [4900, 4901, 4902, ..., 4947, 4948, 4949],\n",
       "       [4950, 4951, 4952, ..., 4997, 4998, 4999]])"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "data=np.arange(5000).reshape(100,50)\n",
    "data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Not so user friendly to look and observer the data and make any inference of it"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Python has long been great for data munging and preparation, but less so for data analysis and modeling. pandas helps fill this gap, enabling you to carry out your entire data analysis workflow in Python.\n",
    "\n",
    "Pandas is a library providing high-performance, easy-to-use data structures and data analysis tools for the Python programming language. Highly optimized for performance, with critical code paths written in Cython or C.\n",
    "\n",
    "\n",
    "source: \n",
    "- https://pandas.pydata.org/\n",
    "- https://pandas.pydata.org/pandas-docs/stable/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Advantages of Pandas"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- A fast and efficient DataFrame object for data manipulation with integrated indexing\n",
    "- Tools for reading and writing data between in-memory data structures and different formats: CSV and text files, Microsoft Excel, SQL databases, and the fast HDF5 format\n",
    "- Intelligent data alignment and integrated handling of missing data: gain automatic label-based alignment in computations and easily manipulate messy data into an orderly form\n",
    "- Flexible reshaping and pivoting of data sets\n",
    "- Intelligent label-based slicing, fancy indexing, and subsetting of large data sets\n",
    "- High performance merging and joining of data sets;\n",
    "- Support for time-series"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: pandas in c:\\users\\themachine\\.conda\\envs\\python36\\lib\\site-packages (0.25.3)\n",
      "Requirement already satisfied: numpy>=1.13.3 in c:\\users\\themachine\\.conda\\envs\\python36\\lib\\site-packages (from pandas) (1.18.0)\n",
      "Requirement already satisfied: pytz>=2017.2 in c:\\users\\themachine\\.conda\\envs\\python36\\lib\\site-packages (from pandas) (2019.3)\n",
      "Requirement already satisfied: python-dateutil>=2.6.1 in c:\\users\\themachine\\.conda\\envs\\python36\\lib\\site-packages (from pandas) (2.8.1)\n",
      "Requirement already satisfied: six>=1.5 in c:\\users\\themachine\\.conda\\envs\\python36\\lib\\site-packages (from python-dateutil>=2.6.1->pandas) (1.13.0)\n"
     ]
    }
   ],
   "source": [
    "!pip install pandas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Frames"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[   0,    1,    2, ...,   47,   48,   49],\n",
       "       [  50,   51,   52, ...,   97,   98,   99],\n",
       "       [ 100,  101,  102, ...,  147,  148,  149],\n",
       "       ...,\n",
       "       [4850, 4851, 4852, ..., 4897, 4898, 4899],\n",
       "       [4900, 4901, 4902, ..., 4947, 4948, 4949],\n",
       "       [4950, 4951, 4952, ..., 4997, 4998, 4999]])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "columnName=['columns_'+str(i) for i in range(50)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['columns_0',\n",
       " 'columns_1',\n",
       " 'columns_2',\n",
       " 'columns_3',\n",
       " 'columns_4',\n",
       " 'columns_5',\n",
       " 'columns_6',\n",
       " 'columns_7',\n",
       " 'columns_8',\n",
       " 'columns_9',\n",
       " 'columns_10',\n",
       " 'columns_11',\n",
       " 'columns_12',\n",
       " 'columns_13',\n",
       " 'columns_14',\n",
       " 'columns_15',\n",
       " 'columns_16',\n",
       " 'columns_17',\n",
       " 'columns_18',\n",
       " 'columns_19',\n",
       " 'columns_20',\n",
       " 'columns_21',\n",
       " 'columns_22',\n",
       " 'columns_23',\n",
       " 'columns_24',\n",
       " 'columns_25',\n",
       " 'columns_26',\n",
       " 'columns_27',\n",
       " 'columns_28',\n",
       " 'columns_29',\n",
       " 'columns_30',\n",
       " 'columns_31',\n",
       " 'columns_32',\n",
       " 'columns_33',\n",
       " 'columns_34',\n",
       " 'columns_35',\n",
       " 'columns_36',\n",
       " 'columns_37',\n",
       " 'columns_38',\n",
       " 'columns_39',\n",
       " 'columns_40',\n",
       " 'columns_41',\n",
       " 'columns_42',\n",
       " 'columns_43',\n",
       " 'columns_44',\n",
       " 'columns_45',\n",
       " 'columns_46',\n",
       " 'columns_47',\n",
       " 'columns_48',\n",
       " 'columns_49']"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "columnName"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataSamp=pd.DataFrame(data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "      <th>9</th>\n",
       "      <th>...</th>\n",
       "      <th>40</th>\n",
       "      <th>41</th>\n",
       "      <th>42</th>\n",
       "      <th>43</th>\n",
       "      <th>44</th>\n",
       "      <th>45</th>\n",
       "      <th>46</th>\n",
       "      <th>47</th>\n",
       "      <th>48</th>\n",
       "      <th>49</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>5</td>\n",
       "      <td>6</td>\n",
       "      <td>7</td>\n",
       "      <td>8</td>\n",
       "      <td>9</td>\n",
       "      <td>...</td>\n",
       "      <td>40</td>\n",
       "      <td>41</td>\n",
       "      <td>42</td>\n",
       "      <td>43</td>\n",
       "      <td>44</td>\n",
       "      <td>45</td>\n",
       "      <td>46</td>\n",
       "      <td>47</td>\n",
       "      <td>48</td>\n",
       "      <td>49</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>50</td>\n",
       "      <td>51</td>\n",
       "      <td>52</td>\n",
       "      <td>53</td>\n",
       "      <td>54</td>\n",
       "      <td>55</td>\n",
       "      <td>56</td>\n",
       "      <td>57</td>\n",
       "      <td>58</td>\n",
       "      <td>59</td>\n",
       "      <td>...</td>\n",
       "      <td>90</td>\n",
       "      <td>91</td>\n",
       "      <td>92</td>\n",
       "      <td>93</td>\n",
       "      <td>94</td>\n",
       "      <td>95</td>\n",
       "      <td>96</td>\n",
       "      <td>97</td>\n",
       "      <td>98</td>\n",
       "      <td>99</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>100</td>\n",
       "      <td>101</td>\n",
       "      <td>102</td>\n",
       "      <td>103</td>\n",
       "      <td>104</td>\n",
       "      <td>105</td>\n",
       "      <td>106</td>\n",
       "      <td>107</td>\n",
       "      <td>108</td>\n",
       "      <td>109</td>\n",
       "      <td>...</td>\n",
       "      <td>140</td>\n",
       "      <td>141</td>\n",
       "      <td>142</td>\n",
       "      <td>143</td>\n",
       "      <td>144</td>\n",
       "      <td>145</td>\n",
       "      <td>146</td>\n",
       "      <td>147</td>\n",
       "      <td>148</td>\n",
       "      <td>149</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>150</td>\n",
       "      <td>151</td>\n",
       "      <td>152</td>\n",
       "      <td>153</td>\n",
       "      <td>154</td>\n",
       "      <td>155</td>\n",
       "      <td>156</td>\n",
       "      <td>157</td>\n",
       "      <td>158</td>\n",
       "      <td>159</td>\n",
       "      <td>...</td>\n",
       "      <td>190</td>\n",
       "      <td>191</td>\n",
       "      <td>192</td>\n",
       "      <td>193</td>\n",
       "      <td>194</td>\n",
       "      <td>195</td>\n",
       "      <td>196</td>\n",
       "      <td>197</td>\n",
       "      <td>198</td>\n",
       "      <td>199</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>200</td>\n",
       "      <td>201</td>\n",
       "      <td>202</td>\n",
       "      <td>203</td>\n",
       "      <td>204</td>\n",
       "      <td>205</td>\n",
       "      <td>206</td>\n",
       "      <td>207</td>\n",
       "      <td>208</td>\n",
       "      <td>209</td>\n",
       "      <td>...</td>\n",
       "      <td>240</td>\n",
       "      <td>241</td>\n",
       "      <td>242</td>\n",
       "      <td>243</td>\n",
       "      <td>244</td>\n",
       "      <td>245</td>\n",
       "      <td>246</td>\n",
       "      <td>247</td>\n",
       "      <td>248</td>\n",
       "      <td>249</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 50 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "    0    1    2    3    4    5    6    7    8    9   ...   40   41   42   43  \\\n",
       "0    0    1    2    3    4    5    6    7    8    9  ...   40   41   42   43   \n",
       "1   50   51   52   53   54   55   56   57   58   59  ...   90   91   92   93   \n",
       "2  100  101  102  103  104  105  106  107  108  109  ...  140  141  142  143   \n",
       "3  150  151  152  153  154  155  156  157  158  159  ...  190  191  192  193   \n",
       "4  200  201  202  203  204  205  206  207  208  209  ...  240  241  242  243   \n",
       "\n",
       "    44   45   46   47   48   49  \n",
       "0   44   45   46   47   48   49  \n",
       "1   94   95   96   97   98   99  \n",
       "2  144  145  146  147  148  149  \n",
       "3  194  195  196  197  198  199  \n",
       "4  244  245  246  247  248  249  \n",
       "\n",
       "[5 rows x 50 columns]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataSamp.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "pandas.core.frame.DataFrame"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(dataSamp)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0        0\n",
       "1       50\n",
       "2      100\n",
       "3      150\n",
       "4      200\n",
       "      ... \n",
       "95    4750\n",
       "96    4800\n",
       "97    4850\n",
       "98    4900\n",
       "99    4950\n",
       "Name: 0, Length: 100, dtype: int32"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataSamp[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "pandas.core.series.Series"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(dataSamp[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataSamp=pd.DataFrame(data,columns=columnName)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>columns_0</th>\n",
       "      <th>columns_1</th>\n",
       "      <th>columns_2</th>\n",
       "      <th>columns_3</th>\n",
       "      <th>columns_4</th>\n",
       "      <th>columns_5</th>\n",
       "      <th>columns_6</th>\n",
       "      <th>columns_7</th>\n",
       "      <th>columns_8</th>\n",
       "      <th>columns_9</th>\n",
       "      <th>...</th>\n",
       "      <th>columns_40</th>\n",
       "      <th>columns_41</th>\n",
       "      <th>columns_42</th>\n",
       "      <th>columns_43</th>\n",
       "      <th>columns_44</th>\n",
       "      <th>columns_45</th>\n",
       "      <th>columns_46</th>\n",
       "      <th>columns_47</th>\n",
       "      <th>columns_48</th>\n",
       "      <th>columns_49</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>5</td>\n",
       "      <td>6</td>\n",
       "      <td>7</td>\n",
       "      <td>8</td>\n",
       "      <td>9</td>\n",
       "      <td>...</td>\n",
       "      <td>40</td>\n",
       "      <td>41</td>\n",
       "      <td>42</td>\n",
       "      <td>43</td>\n",
       "      <td>44</td>\n",
       "      <td>45</td>\n",
       "      <td>46</td>\n",
       "      <td>47</td>\n",
       "      <td>48</td>\n",
       "      <td>49</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>50</td>\n",
       "      <td>51</td>\n",
       "      <td>52</td>\n",
       "      <td>53</td>\n",
       "      <td>54</td>\n",
       "      <td>55</td>\n",
       "      <td>56</td>\n",
       "      <td>57</td>\n",
       "      <td>58</td>\n",
       "      <td>59</td>\n",
       "      <td>...</td>\n",
       "      <td>90</td>\n",
       "      <td>91</td>\n",
       "      <td>92</td>\n",
       "      <td>93</td>\n",
       "      <td>94</td>\n",
       "      <td>95</td>\n",
       "      <td>96</td>\n",
       "      <td>97</td>\n",
       "      <td>98</td>\n",
       "      <td>99</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>100</td>\n",
       "      <td>101</td>\n",
       "      <td>102</td>\n",
       "      <td>103</td>\n",
       "      <td>104</td>\n",
       "      <td>105</td>\n",
       "      <td>106</td>\n",
       "      <td>107</td>\n",
       "      <td>108</td>\n",
       "      <td>109</td>\n",
       "      <td>...</td>\n",
       "      <td>140</td>\n",
       "      <td>141</td>\n",
       "      <td>142</td>\n",
       "      <td>143</td>\n",
       "      <td>144</td>\n",
       "      <td>145</td>\n",
       "      <td>146</td>\n",
       "      <td>147</td>\n",
       "      <td>148</td>\n",
       "      <td>149</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>150</td>\n",
       "      <td>151</td>\n",
       "      <td>152</td>\n",
       "      <td>153</td>\n",
       "      <td>154</td>\n",
       "      <td>155</td>\n",
       "      <td>156</td>\n",
       "      <td>157</td>\n",
       "      <td>158</td>\n",
       "      <td>159</td>\n",
       "      <td>...</td>\n",
       "      <td>190</td>\n",
       "      <td>191</td>\n",
       "      <td>192</td>\n",
       "      <td>193</td>\n",
       "      <td>194</td>\n",
       "      <td>195</td>\n",
       "      <td>196</td>\n",
       "      <td>197</td>\n",
       "      <td>198</td>\n",
       "      <td>199</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>200</td>\n",
       "      <td>201</td>\n",
       "      <td>202</td>\n",
       "      <td>203</td>\n",
       "      <td>204</td>\n",
       "      <td>205</td>\n",
       "      <td>206</td>\n",
       "      <td>207</td>\n",
       "      <td>208</td>\n",
       "      <td>209</td>\n",
       "      <td>...</td>\n",
       "      <td>240</td>\n",
       "      <td>241</td>\n",
       "      <td>242</td>\n",
       "      <td>243</td>\n",
       "      <td>244</td>\n",
       "      <td>245</td>\n",
       "      <td>246</td>\n",
       "      <td>247</td>\n",
       "      <td>248</td>\n",
       "      <td>249</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 50 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   columns_0  columns_1  columns_2  columns_3  columns_4  columns_5  \\\n",
       "0          0          1          2          3          4          5   \n",
       "1         50         51         52         53         54         55   \n",
       "2        100        101        102        103        104        105   \n",
       "3        150        151        152        153        154        155   \n",
       "4        200        201        202        203        204        205   \n",
       "\n",
       "   columns_6  columns_7  columns_8  columns_9  ...  columns_40  columns_41  \\\n",
       "0          6          7          8          9  ...          40          41   \n",
       "1         56         57         58         59  ...          90          91   \n",
       "2        106        107        108        109  ...         140         141   \n",
       "3        156        157        158        159  ...         190         191   \n",
       "4        206        207        208        209  ...         240         241   \n",
       "\n",
       "   columns_42  columns_43  columns_44  columns_45  columns_46  columns_47  \\\n",
       "0          42          43          44          45          46          47   \n",
       "1          92          93          94          95          96          97   \n",
       "2         142         143         144         145         146         147   \n",
       "3         192         193         194         195         196         197   \n",
       "4         242         243         244         245         246         247   \n",
       "\n",
       "   columns_48  columns_49  \n",
       "0          48          49  \n",
       "1          98          99  \n",
       "2         148         149  \n",
       "3         198         199  \n",
       "4         248         249  \n",
       "\n",
       "[5 rows x 50 columns]"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataSamp.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "dict1={'name1': 'XYZ1','name2':'XYZ2'}\n",
    "dict2={'score1': 15,'score2':15}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>name1</th>\n",
       "      <th>name2</th>\n",
       "      <th>score1</th>\n",
       "      <th>score2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>XYZ1</td>\n",
       "      <td>XYZ2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>15.0</td>\n",
       "      <td>15.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  name1 name2  score1  score2\n",
       "0  XYZ1  XYZ2     NaN     NaN\n",
       "1   NaN   NaN    15.0    15.0"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.DataFrame(data=(dict1,dict2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "dict1={'name1': list(range(10)),'name2':list(range(10,20))}\n",
    "# dict2={'score1': list(range(150,200)),'score2':list(range(100,150))}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>name1</th>\n",
       "      <th>name2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>5</td>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>6</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>7</td>\n",
       "      <td>17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>8</td>\n",
       "      <td>18</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>9</td>\n",
       "      <td>19</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   name1  name2\n",
       "0      0     10\n",
       "1      1     11\n",
       "2      2     12\n",
       "3      3     13\n",
       "4      4     14\n",
       "5      5     15\n",
       "6      6     16\n",
       "7      7     17\n",
       "8      8     18\n",
       "9      9     19"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.DataFrame(data=(dict1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Read data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "data=pd.read_csv('dataset_32_pendigits.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>input1</th>\n",
       "      <th>input2</th>\n",
       "      <th>input3</th>\n",
       "      <th>input4</th>\n",
       "      <th>input5</th>\n",
       "      <th>input6</th>\n",
       "      <th>input7</th>\n",
       "      <th>input8</th>\n",
       "      <th>input9</th>\n",
       "      <th>input10</th>\n",
       "      <th>input11</th>\n",
       "      <th>input12</th>\n",
       "      <th>input13</th>\n",
       "      <th>input14</th>\n",
       "      <th>input15</th>\n",
       "      <th>input16</th>\n",
       "      <th>class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>47</td>\n",
       "      <td>100</td>\n",
       "      <td>27</td>\n",
       "      <td>81</td>\n",
       "      <td>57</td>\n",
       "      <td>37</td>\n",
       "      <td>26</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>23</td>\n",
       "      <td>56.0</td>\n",
       "      <td>53</td>\n",
       "      <td>100</td>\n",
       "      <td>90</td>\n",
       "      <td>40</td>\n",
       "      <td>98</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>89</td>\n",
       "      <td>27</td>\n",
       "      <td>100</td>\n",
       "      <td>42</td>\n",
       "      <td>75</td>\n",
       "      <td>29</td>\n",
       "      <td>45</td>\n",
       "      <td>15</td>\n",
       "      <td>15</td>\n",
       "      <td>37.0</td>\n",
       "      <td>0</td>\n",
       "      <td>69</td>\n",
       "      <td>2</td>\n",
       "      <td>100</td>\n",
       "      <td>6</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>57</td>\n",
       "      <td>31</td>\n",
       "      <td>68</td>\n",
       "      <td>72</td>\n",
       "      <td>90</td>\n",
       "      <td>100</td>\n",
       "      <td>100</td>\n",
       "      <td>76</td>\n",
       "      <td>75</td>\n",
       "      <td>50.0</td>\n",
       "      <td>51</td>\n",
       "      <td>28</td>\n",
       "      <td>25</td>\n",
       "      <td>16</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>100</td>\n",
       "      <td>7</td>\n",
       "      <td>92</td>\n",
       "      <td>5</td>\n",
       "      <td>68</td>\n",
       "      <td>19</td>\n",
       "      <td>45</td>\n",
       "      <td>86</td>\n",
       "      <td>34</td>\n",
       "      <td>100.0</td>\n",
       "      <td>45</td>\n",
       "      <td>74</td>\n",
       "      <td>23</td>\n",
       "      <td>67</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>67</td>\n",
       "      <td>49</td>\n",
       "      <td>83</td>\n",
       "      <td>100</td>\n",
       "      <td>100</td>\n",
       "      <td>81</td>\n",
       "      <td>80</td>\n",
       "      <td>60</td>\n",
       "      <td>60</td>\n",
       "      <td>40.0</td>\n",
       "      <td>40</td>\n",
       "      <td>33</td>\n",
       "      <td>20</td>\n",
       "      <td>47</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   input1  input2  input3  input4  input5  input6  input7  input8  input9  \\\n",
       "0      47     100      27      81      57      37      26       0       0   \n",
       "1       0      89      27     100      42      75      29      45      15   \n",
       "2       0      57      31      68      72      90     100     100      76   \n",
       "3       0     100       7      92       5      68      19      45      86   \n",
       "4       0      67      49      83     100     100      81      80      60   \n",
       "\n",
       "   input10  input11  input12  input13  input14  input15  input16  class  \n",
       "0       23     56.0       53      100       90       40       98      8  \n",
       "1       15     37.0        0       69        2      100        6      2  \n",
       "2       75     50.0       51       28       25       16        0      1  \n",
       "3       34    100.0       45       74       23       67        0      4  \n",
       "4       60     40.0       40       33       20       47        0      1  "
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.head() #top 5 default, we can choose our own numbers of row to visualize"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>input1</th>\n",
       "      <th>input2</th>\n",
       "      <th>input3</th>\n",
       "      <th>input4</th>\n",
       "      <th>input5</th>\n",
       "      <th>input6</th>\n",
       "      <th>input7</th>\n",
       "      <th>input8</th>\n",
       "      <th>input9</th>\n",
       "      <th>input10</th>\n",
       "      <th>input11</th>\n",
       "      <th>input12</th>\n",
       "      <th>input13</th>\n",
       "      <th>input14</th>\n",
       "      <th>input15</th>\n",
       "      <th>input16</th>\n",
       "      <th>class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>10987</th>\n",
       "      <td>36</td>\n",
       "      <td>100</td>\n",
       "      <td>24</td>\n",
       "      <td>70</td>\n",
       "      <td>0</td>\n",
       "      <td>38</td>\n",
       "      <td>49</td>\n",
       "      <td>33</td>\n",
       "      <td>95</td>\n",
       "      <td>47</td>\n",
       "      <td>87.0</td>\n",
       "      <td>55</td>\n",
       "      <td>96</td>\n",
       "      <td>21</td>\n",
       "      <td>100</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10988</th>\n",
       "      <td>16</td>\n",
       "      <td>75</td>\n",
       "      <td>41</td>\n",
       "      <td>100</td>\n",
       "      <td>52</td>\n",
       "      <td>64</td>\n",
       "      <td>32</td>\n",
       "      <td>27</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>21.0</td>\n",
       "      <td>9</td>\n",
       "      <td>62</td>\n",
       "      <td>2</td>\n",
       "      <td>100</td>\n",
       "      <td>14</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10989</th>\n",
       "      <td>56</td>\n",
       "      <td>100</td>\n",
       "      <td>27</td>\n",
       "      <td>79</td>\n",
       "      <td>0</td>\n",
       "      <td>39</td>\n",
       "      <td>12</td>\n",
       "      <td>0</td>\n",
       "      <td>66</td>\n",
       "      <td>15</td>\n",
       "      <td>100.0</td>\n",
       "      <td>51</td>\n",
       "      <td>93</td>\n",
       "      <td>93</td>\n",
       "      <td>38</td>\n",
       "      <td>93</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10990</th>\n",
       "      <td>19</td>\n",
       "      <td>100</td>\n",
       "      <td>0</td>\n",
       "      <td>61</td>\n",
       "      <td>3</td>\n",
       "      <td>23</td>\n",
       "      <td>48</td>\n",
       "      <td>0</td>\n",
       "      <td>97</td>\n",
       "      <td>27</td>\n",
       "      <td>100.0</td>\n",
       "      <td>66</td>\n",
       "      <td>62</td>\n",
       "      <td>97</td>\n",
       "      <td>10</td>\n",
       "      <td>81</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10991</th>\n",
       "      <td>38</td>\n",
       "      <td>100</td>\n",
       "      <td>37</td>\n",
       "      <td>81</td>\n",
       "      <td>12</td>\n",
       "      <td>55</td>\n",
       "      <td>0</td>\n",
       "      <td>28</td>\n",
       "      <td>52</td>\n",
       "      <td>27</td>\n",
       "      <td>100.0</td>\n",
       "      <td>42</td>\n",
       "      <td>86</td>\n",
       "      <td>26</td>\n",
       "      <td>65</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       input1  input2  input3  input4  input5  input6  input7  input8  input9  \\\n",
       "10987      36     100      24      70       0      38      49      33      95   \n",
       "10988      16      75      41     100      52      64      32      27       0   \n",
       "10989      56     100      27      79       0      39      12       0      66   \n",
       "10990      19     100       0      61       3      23      48       0      97   \n",
       "10991      38     100      37      81      12      55       0      28      52   \n",
       "\n",
       "       input10  input11  input12  input13  input14  input15  input16  class  \n",
       "10987       47     87.0       55       96       21      100        0      4  \n",
       "10988        0     21.0        9       62        2      100       14      2  \n",
       "10989       15    100.0       51       93       93       38       93      0  \n",
       "10990       27    100.0       66       62       97       10       81      0  \n",
       "10991       27    100.0       42       86       26       65        0      4  "
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.tail()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['input1', 'input2', 'input3', 'input4', 'input5', 'input6', 'input7',\n",
       "       'input8', 'input9', 'input10', 'input11', 'input12', 'input13',\n",
       "       'input14', 'input15', 'input16', 'class'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "input1       int64\n",
       "input2       int64\n",
       "input3       int64\n",
       "input4       int64\n",
       "input5       int64\n",
       "input6       int64\n",
       "input7       int64\n",
       "input8       int64\n",
       "input9       int64\n",
       "input10      int64\n",
       "input11    float64\n",
       "input12      int64\n",
       "input13      int64\n",
       "input14      int64\n",
       "input15      int64\n",
       "input16      int64\n",
       "class        int64\n",
       "dtype: object"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.dtypes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>input1</th>\n",
       "      <th>input2</th>\n",
       "      <th>input3</th>\n",
       "      <th>input4</th>\n",
       "      <th>input5</th>\n",
       "      <th>input6</th>\n",
       "      <th>input7</th>\n",
       "      <th>input8</th>\n",
       "      <th>input9</th>\n",
       "      <th>input10</th>\n",
       "      <th>input11</th>\n",
       "      <th>input12</th>\n",
       "      <th>input13</th>\n",
       "      <th>input14</th>\n",
       "      <th>input15</th>\n",
       "      <th>input16</th>\n",
       "      <th>class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>10992.000000</td>\n",
       "      <td>10992.000000</td>\n",
       "      <td>10992.000000</td>\n",
       "      <td>10992.000000</td>\n",
       "      <td>10992.000000</td>\n",
       "      <td>10992.000000</td>\n",
       "      <td>10992.000000</td>\n",
       "      <td>10992.000000</td>\n",
       "      <td>10992.000000</td>\n",
       "      <td>10992.000000</td>\n",
       "      <td>10744.000000</td>\n",
       "      <td>10992.000000</td>\n",
       "      <td>10992.000000</td>\n",
       "      <td>10992.000000</td>\n",
       "      <td>10992.000000</td>\n",
       "      <td>10992.000000</td>\n",
       "      <td>10992.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>38.814320</td>\n",
       "      <td>85.120269</td>\n",
       "      <td>40.605622</td>\n",
       "      <td>83.774199</td>\n",
       "      <td>49.770378</td>\n",
       "      <td>65.573144</td>\n",
       "      <td>51.220251</td>\n",
       "      <td>44.498999</td>\n",
       "      <td>56.868541</td>\n",
       "      <td>33.695961</td>\n",
       "      <td>59.769918</td>\n",
       "      <td>34.826510</td>\n",
       "      <td>55.022289</td>\n",
       "      <td>34.937045</td>\n",
       "      <td>47.287482</td>\n",
       "      <td>28.845342</td>\n",
       "      <td>4.431587</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>34.257783</td>\n",
       "      <td>16.218571</td>\n",
       "      <td>26.342984</td>\n",
       "      <td>19.163646</td>\n",
       "      <td>34.100515</td>\n",
       "      <td>26.996688</td>\n",
       "      <td>30.576881</td>\n",
       "      <td>29.906104</td>\n",
       "      <td>34.135530</td>\n",
       "      <td>27.251548</td>\n",
       "      <td>37.385931</td>\n",
       "      <td>27.119982</td>\n",
       "      <td>22.335539</td>\n",
       "      <td>33.155463</td>\n",
       "      <td>41.760400</td>\n",
       "      <td>35.778094</td>\n",
       "      <td>2.876947</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>6.000000</td>\n",
       "      <td>76.000000</td>\n",
       "      <td>20.000000</td>\n",
       "      <td>72.000000</td>\n",
       "      <td>18.000000</td>\n",
       "      <td>49.000000</td>\n",
       "      <td>28.000000</td>\n",
       "      <td>23.000000</td>\n",
       "      <td>29.000000</td>\n",
       "      <td>7.000000</td>\n",
       "      <td>22.000000</td>\n",
       "      <td>11.000000</td>\n",
       "      <td>42.000000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>2.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>32.000000</td>\n",
       "      <td>89.000000</td>\n",
       "      <td>40.000000</td>\n",
       "      <td>91.000000</td>\n",
       "      <td>53.000000</td>\n",
       "      <td>71.000000</td>\n",
       "      <td>53.500000</td>\n",
       "      <td>43.000000</td>\n",
       "      <td>60.000000</td>\n",
       "      <td>33.000000</td>\n",
       "      <td>72.000000</td>\n",
       "      <td>30.000000</td>\n",
       "      <td>53.000000</td>\n",
       "      <td>27.000000</td>\n",
       "      <td>40.000000</td>\n",
       "      <td>9.000000</td>\n",
       "      <td>4.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>65.000000</td>\n",
       "      <td>100.000000</td>\n",
       "      <td>58.000000</td>\n",
       "      <td>100.000000</td>\n",
       "      <td>78.000000</td>\n",
       "      <td>86.000000</td>\n",
       "      <td>74.000000</td>\n",
       "      <td>64.000000</td>\n",
       "      <td>89.000000</td>\n",
       "      <td>54.000000</td>\n",
       "      <td>98.000000</td>\n",
       "      <td>55.000000</td>\n",
       "      <td>68.000000</td>\n",
       "      <td>47.000000</td>\n",
       "      <td>100.000000</td>\n",
       "      <td>51.000000</td>\n",
       "      <td>7.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>100.000000</td>\n",
       "      <td>100.000000</td>\n",
       "      <td>100.000000</td>\n",
       "      <td>100.000000</td>\n",
       "      <td>100.000000</td>\n",
       "      <td>100.000000</td>\n",
       "      <td>100.000000</td>\n",
       "      <td>100.000000</td>\n",
       "      <td>100.000000</td>\n",
       "      <td>100.000000</td>\n",
       "      <td>100.000000</td>\n",
       "      <td>100.000000</td>\n",
       "      <td>100.000000</td>\n",
       "      <td>100.000000</td>\n",
       "      <td>100.000000</td>\n",
       "      <td>100.000000</td>\n",
       "      <td>9.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             input1        input2        input3        input4        input5  \\\n",
       "count  10992.000000  10992.000000  10992.000000  10992.000000  10992.000000   \n",
       "mean      38.814320     85.120269     40.605622     83.774199     49.770378   \n",
       "std       34.257783     16.218571     26.342984     19.163646     34.100515   \n",
       "min        0.000000      0.000000      0.000000      0.000000      0.000000   \n",
       "25%        6.000000     76.000000     20.000000     72.000000     18.000000   \n",
       "50%       32.000000     89.000000     40.000000     91.000000     53.000000   \n",
       "75%       65.000000    100.000000     58.000000    100.000000     78.000000   \n",
       "max      100.000000    100.000000    100.000000    100.000000    100.000000   \n",
       "\n",
       "             input6        input7        input8        input9       input10  \\\n",
       "count  10992.000000  10992.000000  10992.000000  10992.000000  10992.000000   \n",
       "mean      65.573144     51.220251     44.498999     56.868541     33.695961   \n",
       "std       26.996688     30.576881     29.906104     34.135530     27.251548   \n",
       "min        0.000000      0.000000      0.000000      0.000000      0.000000   \n",
       "25%       49.000000     28.000000     23.000000     29.000000      7.000000   \n",
       "50%       71.000000     53.500000     43.000000     60.000000     33.000000   \n",
       "75%       86.000000     74.000000     64.000000     89.000000     54.000000   \n",
       "max      100.000000    100.000000    100.000000    100.000000    100.000000   \n",
       "\n",
       "            input11       input12       input13       input14       input15  \\\n",
       "count  10744.000000  10992.000000  10992.000000  10992.000000  10992.000000   \n",
       "mean      59.769918     34.826510     55.022289     34.937045     47.287482   \n",
       "std       37.385931     27.119982     22.335539     33.155463     41.760400   \n",
       "min        0.000000      0.000000      0.000000      0.000000      0.000000   \n",
       "25%       22.000000     11.000000     42.000000      5.000000      0.000000   \n",
       "50%       72.000000     30.000000     53.000000     27.000000     40.000000   \n",
       "75%       98.000000     55.000000     68.000000     47.000000    100.000000   \n",
       "max      100.000000    100.000000    100.000000    100.000000    100.000000   \n",
       "\n",
       "            input16         class  \n",
       "count  10992.000000  10992.000000  \n",
       "mean      28.845342      4.431587  \n",
       "std       35.778094      2.876947  \n",
       "min        0.000000      0.000000  \n",
       "25%        0.000000      2.000000  \n",
       "50%        9.000000      4.000000  \n",
       "75%       51.000000      7.000000  \n",
       "max      100.000000      9.000000  "
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RangeIndex(start=0, stop=10992, step=1)"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>input1</th>\n",
       "      <th>input2</th>\n",
       "      <th>input3</th>\n",
       "      <th>input4</th>\n",
       "      <th>input5</th>\n",
       "      <th>input6</th>\n",
       "      <th>input7</th>\n",
       "      <th>input8</th>\n",
       "      <th>input9</th>\n",
       "      <th>input10</th>\n",
       "      <th>input11</th>\n",
       "      <th>input12</th>\n",
       "      <th>input13</th>\n",
       "      <th>input14</th>\n",
       "      <th>input15</th>\n",
       "      <th>input16</th>\n",
       "      <th>class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>47</td>\n",
       "      <td>100</td>\n",
       "      <td>27</td>\n",
       "      <td>81</td>\n",
       "      <td>57</td>\n",
       "      <td>37</td>\n",
       "      <td>26</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>23</td>\n",
       "      <td>56.0</td>\n",
       "      <td>53</td>\n",
       "      <td>100</td>\n",
       "      <td>90</td>\n",
       "      <td>40</td>\n",
       "      <td>98</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>89</td>\n",
       "      <td>27</td>\n",
       "      <td>100</td>\n",
       "      <td>42</td>\n",
       "      <td>75</td>\n",
       "      <td>29</td>\n",
       "      <td>45</td>\n",
       "      <td>15</td>\n",
       "      <td>15</td>\n",
       "      <td>37.0</td>\n",
       "      <td>0</td>\n",
       "      <td>69</td>\n",
       "      <td>2</td>\n",
       "      <td>100</td>\n",
       "      <td>6</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>57</td>\n",
       "      <td>31</td>\n",
       "      <td>68</td>\n",
       "      <td>72</td>\n",
       "      <td>90</td>\n",
       "      <td>100</td>\n",
       "      <td>100</td>\n",
       "      <td>76</td>\n",
       "      <td>75</td>\n",
       "      <td>50.0</td>\n",
       "      <td>51</td>\n",
       "      <td>28</td>\n",
       "      <td>25</td>\n",
       "      <td>16</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>100</td>\n",
       "      <td>7</td>\n",
       "      <td>92</td>\n",
       "      <td>5</td>\n",
       "      <td>68</td>\n",
       "      <td>19</td>\n",
       "      <td>45</td>\n",
       "      <td>86</td>\n",
       "      <td>34</td>\n",
       "      <td>100.0</td>\n",
       "      <td>45</td>\n",
       "      <td>74</td>\n",
       "      <td>23</td>\n",
       "      <td>67</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>67</td>\n",
       "      <td>49</td>\n",
       "      <td>83</td>\n",
       "      <td>100</td>\n",
       "      <td>100</td>\n",
       "      <td>81</td>\n",
       "      <td>80</td>\n",
       "      <td>60</td>\n",
       "      <td>60</td>\n",
       "      <td>40.0</td>\n",
       "      <td>40</td>\n",
       "      <td>33</td>\n",
       "      <td>20</td>\n",
       "      <td>47</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   input1  input2  input3  input4  input5  input6  input7  input8  input9  \\\n",
       "0      47     100      27      81      57      37      26       0       0   \n",
       "1       0      89      27     100      42      75      29      45      15   \n",
       "2       0      57      31      68      72      90     100     100      76   \n",
       "3       0     100       7      92       5      68      19      45      86   \n",
       "4       0      67      49      83     100     100      81      80      60   \n",
       "\n",
       "   input10  input11  input12  input13  input14  input15  input16  class  \n",
       "0       23     56.0       53      100       90       40       98      8  \n",
       "1       15     37.0        0       69        2      100        6      2  \n",
       "2       75     50.0       51       28       25       16        0      1  \n",
       "3       34    100.0       45       74       23       67        0      4  \n",
       "4       60     40.0       40       33       20       47        0      1  "
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Selecting columns and slicing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### selection by labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0        47\n",
       "1         0\n",
       "2         0\n",
       "3         0\n",
       "4         0\n",
       "         ..\n",
       "10987    36\n",
       "10988    16\n",
       "10989    56\n",
       "10990    19\n",
       "10991    38\n",
       "Name: input1, Length: 10992, dtype: int64"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.input1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "pandas.core.series.Series"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(data.input1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0        47\n",
       "1         0\n",
       "2         0\n",
       "3         0\n",
       "4         0\n",
       "         ..\n",
       "10987    36\n",
       "10988    16\n",
       "10989    56\n",
       "10990    19\n",
       "10991    38\n",
       "Name: input1, Length: 10992, dtype: int64"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data['input1']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "pandas.core.series.Series"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(data['input1'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>input1</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>47</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10987</th>\n",
       "      <td>36</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10988</th>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10989</th>\n",
       "      <td>56</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10990</th>\n",
       "      <td>19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10991</th>\n",
       "      <td>38</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>10992 rows × 1 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       input1\n",
       "0          47\n",
       "1           0\n",
       "2           0\n",
       "3           0\n",
       "4           0\n",
       "...       ...\n",
       "10987      36\n",
       "10988      16\n",
       "10989      56\n",
       "10990      19\n",
       "10991      38\n",
       "\n",
       "[10992 rows x 1 columns]"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[['input1']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "pandas.core.frame.DataFrame"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(data[['input1']])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>input1</th>\n",
       "      <th>input2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>47</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>89</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>57</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>67</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10987</th>\n",
       "      <td>36</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10988</th>\n",
       "      <td>16</td>\n",
       "      <td>75</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10989</th>\n",
       "      <td>56</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10990</th>\n",
       "      <td>19</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10991</th>\n",
       "      <td>38</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>10992 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       input1  input2\n",
       "0          47     100\n",
       "1           0      89\n",
       "2           0      57\n",
       "3           0     100\n",
       "4           0      67\n",
       "...       ...     ...\n",
       "10987      36     100\n",
       "10988      16      75\n",
       "10989      56     100\n",
       "10990      19     100\n",
       "10991      38     100\n",
       "\n",
       "[10992 rows x 2 columns]"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[['input1','input2']]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### [ ] slices the row"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>input1</th>\n",
       "      <th>input2</th>\n",
       "      <th>input3</th>\n",
       "      <th>input4</th>\n",
       "      <th>input5</th>\n",
       "      <th>input6</th>\n",
       "      <th>input7</th>\n",
       "      <th>input8</th>\n",
       "      <th>input9</th>\n",
       "      <th>input10</th>\n",
       "      <th>input11</th>\n",
       "      <th>input12</th>\n",
       "      <th>input13</th>\n",
       "      <th>input14</th>\n",
       "      <th>input15</th>\n",
       "      <th>input16</th>\n",
       "      <th>class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>47</td>\n",
       "      <td>100</td>\n",
       "      <td>27</td>\n",
       "      <td>81</td>\n",
       "      <td>57</td>\n",
       "      <td>37</td>\n",
       "      <td>26</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>23</td>\n",
       "      <td>56.0</td>\n",
       "      <td>53</td>\n",
       "      <td>100</td>\n",
       "      <td>90</td>\n",
       "      <td>40</td>\n",
       "      <td>98</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>89</td>\n",
       "      <td>27</td>\n",
       "      <td>100</td>\n",
       "      <td>42</td>\n",
       "      <td>75</td>\n",
       "      <td>29</td>\n",
       "      <td>45</td>\n",
       "      <td>15</td>\n",
       "      <td>15</td>\n",
       "      <td>37.0</td>\n",
       "      <td>0</td>\n",
       "      <td>69</td>\n",
       "      <td>2</td>\n",
       "      <td>100</td>\n",
       "      <td>6</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>57</td>\n",
       "      <td>31</td>\n",
       "      <td>68</td>\n",
       "      <td>72</td>\n",
       "      <td>90</td>\n",
       "      <td>100</td>\n",
       "      <td>100</td>\n",
       "      <td>76</td>\n",
       "      <td>75</td>\n",
       "      <td>50.0</td>\n",
       "      <td>51</td>\n",
       "      <td>28</td>\n",
       "      <td>25</td>\n",
       "      <td>16</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>100</td>\n",
       "      <td>7</td>\n",
       "      <td>92</td>\n",
       "      <td>5</td>\n",
       "      <td>68</td>\n",
       "      <td>19</td>\n",
       "      <td>45</td>\n",
       "      <td>86</td>\n",
       "      <td>34</td>\n",
       "      <td>100.0</td>\n",
       "      <td>45</td>\n",
       "      <td>74</td>\n",
       "      <td>23</td>\n",
       "      <td>67</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   input1  input2  input3  input4  input5  input6  input7  input8  input9  \\\n",
       "0      47     100      27      81      57      37      26       0       0   \n",
       "1       0      89      27     100      42      75      29      45      15   \n",
       "2       0      57      31      68      72      90     100     100      76   \n",
       "3       0     100       7      92       5      68      19      45      86   \n",
       "\n",
       "   input10  input11  input12  input13  input14  input15  input16  class  \n",
       "0       23     56.0       53      100       90       40       98      8  \n",
       "1       15     37.0        0       69        2      100        6      2  \n",
       "2       75     50.0       51       28       25       16        0      1  \n",
       "3       34    100.0       45       74       23       67        0      4  "
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[:4]  # works on index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>input1</th>\n",
       "      <th>input2</th>\n",
       "      <th>input3</th>\n",
       "      <th>input4</th>\n",
       "      <th>input5</th>\n",
       "      <th>input6</th>\n",
       "      <th>input7</th>\n",
       "      <th>input8</th>\n",
       "      <th>input9</th>\n",
       "      <th>input10</th>\n",
       "      <th>input11</th>\n",
       "      <th>input12</th>\n",
       "      <th>input13</th>\n",
       "      <th>input14</th>\n",
       "      <th>input15</th>\n",
       "      <th>input16</th>\n",
       "      <th>class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>0</td>\n",
       "      <td>39</td>\n",
       "      <td>2</td>\n",
       "      <td>62</td>\n",
       "      <td>11</td>\n",
       "      <td>5</td>\n",
       "      <td>63</td>\n",
       "      <td>0</td>\n",
       "      <td>100</td>\n",
       "      <td>43</td>\n",
       "      <td>89.0</td>\n",
       "      <td>99</td>\n",
       "      <td>36</td>\n",
       "      <td>100</td>\n",
       "      <td>0</td>\n",
       "      <td>57</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>13</td>\n",
       "      <td>89</td>\n",
       "      <td>12</td>\n",
       "      <td>50</td>\n",
       "      <td>72</td>\n",
       "      <td>38</td>\n",
       "      <td>56</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>17</td>\n",
       "      <td>0.0</td>\n",
       "      <td>61</td>\n",
       "      <td>32</td>\n",
       "      <td>94</td>\n",
       "      <td>100</td>\n",
       "      <td>100</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>57</td>\n",
       "      <td>100</td>\n",
       "      <td>22</td>\n",
       "      <td>72</td>\n",
       "      <td>0</td>\n",
       "      <td>31</td>\n",
       "      <td>25</td>\n",
       "      <td>0</td>\n",
       "      <td>75</td>\n",
       "      <td>13</td>\n",
       "      <td>100.0</td>\n",
       "      <td>50</td>\n",
       "      <td>75</td>\n",
       "      <td>87</td>\n",
       "      <td>26</td>\n",
       "      <td>85</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>74</td>\n",
       "      <td>87</td>\n",
       "      <td>31</td>\n",
       "      <td>100</td>\n",
       "      <td>0</td>\n",
       "      <td>69</td>\n",
       "      <td>62</td>\n",
       "      <td>64</td>\n",
       "      <td>100</td>\n",
       "      <td>79</td>\n",
       "      <td>100.0</td>\n",
       "      <td>38</td>\n",
       "      <td>84</td>\n",
       "      <td>0</td>\n",
       "      <td>18</td>\n",
       "      <td>1</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>48</td>\n",
       "      <td>96</td>\n",
       "      <td>62</td>\n",
       "      <td>65</td>\n",
       "      <td>88</td>\n",
       "      <td>27</td>\n",
       "      <td>21</td>\n",
       "      <td>0</td>\n",
       "      <td>21</td>\n",
       "      <td>33</td>\n",
       "      <td>79.0</td>\n",
       "      <td>67</td>\n",
       "      <td>100</td>\n",
       "      <td>100</td>\n",
       "      <td>0</td>\n",
       "      <td>85</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>100</td>\n",
       "      <td>100</td>\n",
       "      <td>72</td>\n",
       "      <td>99</td>\n",
       "      <td>36</td>\n",
       "      <td>78</td>\n",
       "      <td>34</td>\n",
       "      <td>54</td>\n",
       "      <td>79</td>\n",
       "      <td>47</td>\n",
       "      <td>64.0</td>\n",
       "      <td>13</td>\n",
       "      <td>19</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>91</td>\n",
       "      <td>74</td>\n",
       "      <td>54</td>\n",
       "      <td>100</td>\n",
       "      <td>0</td>\n",
       "      <td>87</td>\n",
       "      <td>23</td>\n",
       "      <td>59</td>\n",
       "      <td>81</td>\n",
       "      <td>67</td>\n",
       "      <td>100.0</td>\n",
       "      <td>39</td>\n",
       "      <td>79</td>\n",
       "      <td>4</td>\n",
       "      <td>21</td>\n",
       "      <td>0</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    input1  input2  input3  input4  input5  input6  input7  input8  input9  \\\n",
       "7        0      39       2      62      11       5      63       0     100   \n",
       "8       13      89      12      50      72      38      56       0       4   \n",
       "9       57     100      22      72       0      31      25       0      75   \n",
       "10      74      87      31     100       0      69      62      64     100   \n",
       "11      48      96      62      65      88      27      21       0      21   \n",
       "12     100     100      72      99      36      78      34      54      79   \n",
       "13      91      74      54     100       0      87      23      59      81   \n",
       "\n",
       "    input10  input11  input12  input13  input14  input15  input16  class  \n",
       "7        43     89.0       99       36      100        0       57      0  \n",
       "8        17      0.0       61       32       94      100      100      5  \n",
       "9        13    100.0       50       75       87       26       85      0  \n",
       "10       79    100.0       38       84        0       18        1      9  \n",
       "11       33     79.0       67      100      100        0       85      8  \n",
       "12       47     64.0       13       19        0        0        2      5  \n",
       "13       67    100.0       39       79        4       21        0      9  "
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[7:14]  # works on index"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Selection by labels"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### .loc   attribute access a group of rows and columns by label(s) or a boolean array in the given DataFrame"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "input1       0.0\n",
       "input2      89.0\n",
       "input3      27.0\n",
       "input4     100.0\n",
       "input5      42.0\n",
       "input6      75.0\n",
       "input7      29.0\n",
       "input8      45.0\n",
       "input9      15.0\n",
       "input10     15.0\n",
       "input11     37.0\n",
       "input12      0.0\n",
       "input13     69.0\n",
       "input14      2.0\n",
       "input15    100.0\n",
       "input16      6.0\n",
       "class        2.0\n",
       "Name: 1, dtype: float64"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.loc[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>input1</th>\n",
       "      <th>input2</th>\n",
       "      <th>input3</th>\n",
       "      <th>input4</th>\n",
       "      <th>input5</th>\n",
       "      <th>input6</th>\n",
       "      <th>input7</th>\n",
       "      <th>input8</th>\n",
       "      <th>input9</th>\n",
       "      <th>input10</th>\n",
       "      <th>input11</th>\n",
       "      <th>input12</th>\n",
       "      <th>input13</th>\n",
       "      <th>input14</th>\n",
       "      <th>input15</th>\n",
       "      <th>input16</th>\n",
       "      <th>class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>89</td>\n",
       "      <td>27</td>\n",
       "      <td>100</td>\n",
       "      <td>42</td>\n",
       "      <td>75</td>\n",
       "      <td>29</td>\n",
       "      <td>45</td>\n",
       "      <td>15</td>\n",
       "      <td>15</td>\n",
       "      <td>37.0</td>\n",
       "      <td>0</td>\n",
       "      <td>69</td>\n",
       "      <td>2</td>\n",
       "      <td>100</td>\n",
       "      <td>6</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   input1  input2  input3  input4  input5  input6  input7  input8  input9  \\\n",
       "1       0      89      27     100      42      75      29      45      15   \n",
       "\n",
       "   input10  input11  input12  input13  input14  input15  input16  class  \n",
       "1       15     37.0        0       69        2      100        6      2  "
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.loc[[1]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>input1</th>\n",
       "      <th>input2</th>\n",
       "      <th>input3</th>\n",
       "      <th>input4</th>\n",
       "      <th>input5</th>\n",
       "      <th>input6</th>\n",
       "      <th>input7</th>\n",
       "      <th>input8</th>\n",
       "      <th>input9</th>\n",
       "      <th>input10</th>\n",
       "      <th>input11</th>\n",
       "      <th>input12</th>\n",
       "      <th>input13</th>\n",
       "      <th>input14</th>\n",
       "      <th>input15</th>\n",
       "      <th>input16</th>\n",
       "      <th>class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>47</td>\n",
       "      <td>100</td>\n",
       "      <td>27</td>\n",
       "      <td>81</td>\n",
       "      <td>57</td>\n",
       "      <td>37</td>\n",
       "      <td>26</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>23</td>\n",
       "      <td>56.0</td>\n",
       "      <td>53</td>\n",
       "      <td>100</td>\n",
       "      <td>90</td>\n",
       "      <td>40</td>\n",
       "      <td>98</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>89</td>\n",
       "      <td>27</td>\n",
       "      <td>100</td>\n",
       "      <td>42</td>\n",
       "      <td>75</td>\n",
       "      <td>29</td>\n",
       "      <td>45</td>\n",
       "      <td>15</td>\n",
       "      <td>15</td>\n",
       "      <td>37.0</td>\n",
       "      <td>0</td>\n",
       "      <td>69</td>\n",
       "      <td>2</td>\n",
       "      <td>100</td>\n",
       "      <td>6</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>57</td>\n",
       "      <td>31</td>\n",
       "      <td>68</td>\n",
       "      <td>72</td>\n",
       "      <td>90</td>\n",
       "      <td>100</td>\n",
       "      <td>100</td>\n",
       "      <td>76</td>\n",
       "      <td>75</td>\n",
       "      <td>50.0</td>\n",
       "      <td>51</td>\n",
       "      <td>28</td>\n",
       "      <td>25</td>\n",
       "      <td>16</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>100</td>\n",
       "      <td>7</td>\n",
       "      <td>92</td>\n",
       "      <td>5</td>\n",
       "      <td>68</td>\n",
       "      <td>19</td>\n",
       "      <td>45</td>\n",
       "      <td>86</td>\n",
       "      <td>34</td>\n",
       "      <td>100.0</td>\n",
       "      <td>45</td>\n",
       "      <td>74</td>\n",
       "      <td>23</td>\n",
       "      <td>67</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   input1  input2  input3  input4  input5  input6  input7  input8  input9  \\\n",
       "0      47     100      27      81      57      37      26       0       0   \n",
       "1       0      89      27     100      42      75      29      45      15   \n",
       "2       0      57      31      68      72      90     100     100      76   \n",
       "3       0     100       7      92       5      68      19      45      86   \n",
       "\n",
       "   input10  input11  input12  input13  input14  input15  input16  class  \n",
       "0       23     56.0       53      100       90       40       98      8  \n",
       "1       15     37.0        0       69        2      100        6      2  \n",
       "2       75     50.0       51       28       25       16        0      1  \n",
       "3       34    100.0       45       74       23       67        0      4  "
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.loc[:3]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>input1</th>\n",
       "      <th>input2</th>\n",
       "      <th>input3</th>\n",
       "      <th>input4</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>47</td>\n",
       "      <td>100</td>\n",
       "      <td>27</td>\n",
       "      <td>81</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>89</td>\n",
       "      <td>27</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>57</td>\n",
       "      <td>31</td>\n",
       "      <td>68</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>100</td>\n",
       "      <td>7</td>\n",
       "      <td>92</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   input1  input2  input3  input4\n",
       "0      47     100      27      81\n",
       "1       0      89      27     100\n",
       "2       0      57      31      68\n",
       "3       0     100       7      92"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.loc[:3,['input1','input2','input3','input4']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>input1</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>47</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   input1\n",
       "0      47\n",
       "1       0\n",
       "2       0\n",
       "3       0\n",
       "4       0"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.loc[:4,['input1']]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Selection by Position"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### .iloc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "input1       0.0\n",
       "input2      89.0\n",
       "input3      27.0\n",
       "input4     100.0\n",
       "input5      42.0\n",
       "input6      75.0\n",
       "input7      29.0\n",
       "input8      45.0\n",
       "input9      15.0\n",
       "input10     15.0\n",
       "input11     37.0\n",
       "input12      0.0\n",
       "input13     69.0\n",
       "input14      2.0\n",
       "input15    100.0\n",
       "input16      6.0\n",
       "class        2.0\n",
       "Name: 1, dtype: float64"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.iloc[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>input1</th>\n",
       "      <th>input2</th>\n",
       "      <th>input3</th>\n",
       "      <th>input4</th>\n",
       "      <th>input5</th>\n",
       "      <th>input6</th>\n",
       "      <th>input7</th>\n",
       "      <th>input8</th>\n",
       "      <th>input9</th>\n",
       "      <th>input10</th>\n",
       "      <th>input11</th>\n",
       "      <th>input12</th>\n",
       "      <th>input13</th>\n",
       "      <th>input14</th>\n",
       "      <th>input15</th>\n",
       "      <th>input16</th>\n",
       "      <th>class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>47</td>\n",
       "      <td>100</td>\n",
       "      <td>27</td>\n",
       "      <td>81</td>\n",
       "      <td>57</td>\n",
       "      <td>37</td>\n",
       "      <td>26</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>23</td>\n",
       "      <td>56.0</td>\n",
       "      <td>53</td>\n",
       "      <td>100</td>\n",
       "      <td>90</td>\n",
       "      <td>40</td>\n",
       "      <td>98</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>89</td>\n",
       "      <td>27</td>\n",
       "      <td>100</td>\n",
       "      <td>42</td>\n",
       "      <td>75</td>\n",
       "      <td>29</td>\n",
       "      <td>45</td>\n",
       "      <td>15</td>\n",
       "      <td>15</td>\n",
       "      <td>37.0</td>\n",
       "      <td>0</td>\n",
       "      <td>69</td>\n",
       "      <td>2</td>\n",
       "      <td>100</td>\n",
       "      <td>6</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   input1  input2  input3  input4  input5  input6  input7  input8  input9  \\\n",
       "0      47     100      27      81      57      37      26       0       0   \n",
       "1       0      89      27     100      42      75      29      45      15   \n",
       "\n",
       "   input10  input11  input12  input13  input14  input15  input16  class  \n",
       "0       23     56.0       53      100       90       40       98      8  \n",
       "1       15     37.0        0       69        2      100        6      2  "
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.head(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>input1</th>\n",
       "      <th>input2</th>\n",
       "      <th>input3</th>\n",
       "      <th>input4</th>\n",
       "      <th>input5</th>\n",
       "      <th>input6</th>\n",
       "      <th>input7</th>\n",
       "      <th>input8</th>\n",
       "      <th>input9</th>\n",
       "      <th>input10</th>\n",
       "      <th>input11</th>\n",
       "      <th>input12</th>\n",
       "      <th>input13</th>\n",
       "      <th>input14</th>\n",
       "      <th>input15</th>\n",
       "      <th>input16</th>\n",
       "      <th>class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>89</td>\n",
       "      <td>27</td>\n",
       "      <td>100</td>\n",
       "      <td>42</td>\n",
       "      <td>75</td>\n",
       "      <td>29</td>\n",
       "      <td>45</td>\n",
       "      <td>15</td>\n",
       "      <td>15</td>\n",
       "      <td>37.0</td>\n",
       "      <td>0</td>\n",
       "      <td>69</td>\n",
       "      <td>2</td>\n",
       "      <td>100</td>\n",
       "      <td>6</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   input1  input2  input3  input4  input5  input6  input7  input8  input9  \\\n",
       "1       0      89      27     100      42      75      29      45      15   \n",
       "\n",
       "   input10  input11  input12  input13  input14  input15  input16  class  \n",
       "1       15     37.0        0       69        2      100        6      2  "
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.iloc[[1]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>input1</th>\n",
       "      <th>input2</th>\n",
       "      <th>input3</th>\n",
       "      <th>input4</th>\n",
       "      <th>input5</th>\n",
       "      <th>input6</th>\n",
       "      <th>input7</th>\n",
       "      <th>input8</th>\n",
       "      <th>input9</th>\n",
       "      <th>input10</th>\n",
       "      <th>input11</th>\n",
       "      <th>input12</th>\n",
       "      <th>input13</th>\n",
       "      <th>input14</th>\n",
       "      <th>input15</th>\n",
       "      <th>input16</th>\n",
       "      <th>class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>89</td>\n",
       "      <td>27</td>\n",
       "      <td>100</td>\n",
       "      <td>42</td>\n",
       "      <td>75</td>\n",
       "      <td>29</td>\n",
       "      <td>45</td>\n",
       "      <td>15</td>\n",
       "      <td>15</td>\n",
       "      <td>37.0</td>\n",
       "      <td>0</td>\n",
       "      <td>69</td>\n",
       "      <td>2</td>\n",
       "      <td>100</td>\n",
       "      <td>6</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   input1  input2  input3  input4  input5  input6  input7  input8  input9  \\\n",
       "1       0      89      27     100      42      75      29      45      15   \n",
       "\n",
       "   input10  input11  input12  input13  input14  input15  input16  class  \n",
       "1       15     37.0        0       69        2      100        6      2  "
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.loc[[1]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "ename": "TypeError",
     "evalue": "cannot do slice indexing on <class 'pandas.core.indexes.base.Index'> with these indexers [4] of <class 'int'>",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-54-28d1be18e7ab>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0mdata\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mloc\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;36m4\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;36m4\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[1;32mc:\\users\\themachine\\.conda\\envs\\python36\\lib\\site-packages\\pandas\\core\\indexing.py\u001b[0m in \u001b[0;36m__getitem__\u001b[1;34m(self, key)\u001b[0m\n\u001b[0;32m   1416\u001b[0m                 \u001b[1;32mexcept\u001b[0m \u001b[1;33m(\u001b[0m\u001b[0mKeyError\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mIndexError\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mAttributeError\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   1417\u001b[0m                     \u001b[1;32mpass\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 1418\u001b[1;33m             \u001b[1;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_getitem_tuple\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m   1419\u001b[0m         \u001b[1;32melse\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   1420\u001b[0m             \u001b[1;31m# we by definition only have the 0th axis\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mc:\\users\\themachine\\.conda\\envs\\python36\\lib\\site-packages\\pandas\\core\\indexing.py\u001b[0m in \u001b[0;36m_getitem_tuple\u001b[1;34m(self, tup)\u001b[0m\n\u001b[0;32m    820\u001b[0m                 \u001b[1;32mcontinue\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    821\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 822\u001b[1;33m             \u001b[0mretval\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mgetattr\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mretval\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mname\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_getitem_axis\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0maxis\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mi\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    823\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    824\u001b[0m         \u001b[1;32mreturn\u001b[0m \u001b[0mretval\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mc:\\users\\themachine\\.conda\\envs\\python36\\lib\\site-packages\\pandas\\core\\indexing.py\u001b[0m in \u001b[0;36m_getitem_axis\u001b[1;34m(self, key, axis)\u001b[0m\n\u001b[0;32m   1795\u001b[0m         \u001b[1;32mif\u001b[0m \u001b[0misinstance\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mslice\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   1796\u001b[0m             \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_validate_key\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0maxis\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 1797\u001b[1;33m             \u001b[1;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_get_slice_axis\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0maxis\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0maxis\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m   1798\u001b[0m         \u001b[1;32melif\u001b[0m \u001b[0mcom\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mis_bool_indexer\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   1799\u001b[0m             \u001b[1;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_getbool_axis\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0maxis\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0maxis\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mc:\\users\\themachine\\.conda\\envs\\python36\\lib\\site-packages\\pandas\\core\\indexing.py\u001b[0m in \u001b[0;36m_get_slice_axis\u001b[1;34m(self, slice_obj, axis)\u001b[0m\n\u001b[0;32m   1452\u001b[0m         \u001b[0mlabels\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mobj\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_get_axis\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0maxis\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   1453\u001b[0m         indexer = labels.slice_indexer(\n\u001b[1;32m-> 1454\u001b[1;33m             \u001b[0mslice_obj\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mstart\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mslice_obj\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mstop\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mslice_obj\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mstep\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mkind\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mname\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m   1455\u001b[0m         )\n\u001b[0;32m   1456\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mc:\\users\\themachine\\.conda\\envs\\python36\\lib\\site-packages\\pandas\\core\\indexes\\base.py\u001b[0m in \u001b[0;36mslice_indexer\u001b[1;34m(self, start, end, step, kind)\u001b[0m\n\u001b[0;32m   5032\u001b[0m         \u001b[0mslice\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;36m1\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;36m3\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   5033\u001b[0m         \"\"\"\n\u001b[1;32m-> 5034\u001b[1;33m         \u001b[0mstart_slice\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mend_slice\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mslice_locs\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mstart\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mend\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mstep\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mstep\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mkind\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mkind\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m   5035\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   5036\u001b[0m         \u001b[1;31m# return a slice\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mc:\\users\\themachine\\.conda\\envs\\python36\\lib\\site-packages\\pandas\\core\\indexes\\base.py\u001b[0m in \u001b[0;36mslice_locs\u001b[1;34m(self, start, end, step, kind)\u001b[0m\n\u001b[0;32m   5252\u001b[0m         \u001b[0mend_slice\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;32mNone\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   5253\u001b[0m         \u001b[1;32mif\u001b[0m \u001b[0mend\u001b[0m \u001b[1;32mis\u001b[0m \u001b[1;32mnot\u001b[0m \u001b[1;32mNone\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 5254\u001b[1;33m             \u001b[0mend_slice\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mget_slice_bound\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mend\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;34m\"right\"\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mkind\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m   5255\u001b[0m         \u001b[1;32mif\u001b[0m \u001b[0mend_slice\u001b[0m \u001b[1;32mis\u001b[0m \u001b[1;32mNone\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   5256\u001b[0m             \u001b[0mend_slice\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mlen\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mc:\\users\\themachine\\.conda\\envs\\python36\\lib\\site-packages\\pandas\\core\\indexes\\base.py\u001b[0m in \u001b[0;36mget_slice_bound\u001b[1;34m(self, label, side, kind)\u001b[0m\n\u001b[0;32m   5158\u001b[0m         \u001b[1;31m# For datetime indices label may be a string that has to be converted\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   5159\u001b[0m         \u001b[1;31m# to datetime boundary according to its resolution.\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 5160\u001b[1;33m         \u001b[0mlabel\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_maybe_cast_slice_bound\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mlabel\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mside\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mkind\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m   5161\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   5162\u001b[0m         \u001b[1;31m# we need to look up the label\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mc:\\users\\themachine\\.conda\\envs\\python36\\lib\\site-packages\\pandas\\core\\indexes\\base.py\u001b[0m in \u001b[0;36m_maybe_cast_slice_bound\u001b[1;34m(self, label, side, kind)\u001b[0m\n\u001b[0;32m   5110\u001b[0m         \u001b[1;31m# this is rejected (generally .loc gets you here)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   5111\u001b[0m         \u001b[1;32melif\u001b[0m \u001b[0mis_integer\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mlabel\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 5112\u001b[1;33m             \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_invalid_indexer\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m\"slice\"\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mlabel\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m   5113\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   5114\u001b[0m         \u001b[1;32mreturn\u001b[0m \u001b[0mlabel\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mc:\\users\\themachine\\.conda\\envs\\python36\\lib\\site-packages\\pandas\\core\\indexes\\base.py\u001b[0m in \u001b[0;36m_invalid_indexer\u001b[1;34m(self, form, key)\u001b[0m\n\u001b[0;32m   3338\u001b[0m             \u001b[1;34m\"cannot do {form} indexing on {klass} with these \"\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   3339\u001b[0m             \"indexers [{key}] of {kind}\".format(\n\u001b[1;32m-> 3340\u001b[1;33m                 \u001b[0mform\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mform\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mklass\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mtype\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mkey\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mkey\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mkind\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mtype\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m   3341\u001b[0m             )\n\u001b[0;32m   3342\u001b[0m         )\n",
      "\u001b[1;31mTypeError\u001b[0m: cannot do slice indexing on <class 'pandas.core.indexes.base.Index'> with these indexers [4] of <class 'int'>"
     ]
    }
   ],
   "source": [
    "data.loc[:4,:4]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>input1</th>\n",
       "      <th>input2</th>\n",
       "      <th>input3</th>\n",
       "      <th>input4</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>47</td>\n",
       "      <td>100</td>\n",
       "      <td>27</td>\n",
       "      <td>81</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>89</td>\n",
       "      <td>27</td>\n",
       "      <td>100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>57</td>\n",
       "      <td>31</td>\n",
       "      <td>68</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>100</td>\n",
       "      <td>7</td>\n",
       "      <td>92</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   input1  input2  input3  input4\n",
       "0      47     100      27      81\n",
       "1       0      89      27     100\n",
       "2       0      57      31      68\n",
       "3       0     100       7      92"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.iloc[:4,:4]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Filtering and Selection of data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>input1</th>\n",
       "      <th>input2</th>\n",
       "      <th>input3</th>\n",
       "      <th>input4</th>\n",
       "      <th>input5</th>\n",
       "      <th>input6</th>\n",
       "      <th>input7</th>\n",
       "      <th>input8</th>\n",
       "      <th>input9</th>\n",
       "      <th>input10</th>\n",
       "      <th>input11</th>\n",
       "      <th>input12</th>\n",
       "      <th>input13</th>\n",
       "      <th>input14</th>\n",
       "      <th>input15</th>\n",
       "      <th>input16</th>\n",
       "      <th>class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>47</td>\n",
       "      <td>100</td>\n",
       "      <td>27</td>\n",
       "      <td>81</td>\n",
       "      <td>57</td>\n",
       "      <td>37</td>\n",
       "      <td>26</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>23</td>\n",
       "      <td>56.0</td>\n",
       "      <td>53</td>\n",
       "      <td>100</td>\n",
       "      <td>90</td>\n",
       "      <td>40</td>\n",
       "      <td>98</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>89</td>\n",
       "      <td>27</td>\n",
       "      <td>100</td>\n",
       "      <td>42</td>\n",
       "      <td>75</td>\n",
       "      <td>29</td>\n",
       "      <td>45</td>\n",
       "      <td>15</td>\n",
       "      <td>15</td>\n",
       "      <td>37.0</td>\n",
       "      <td>0</td>\n",
       "      <td>69</td>\n",
       "      <td>2</td>\n",
       "      <td>100</td>\n",
       "      <td>6</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>57</td>\n",
       "      <td>31</td>\n",
       "      <td>68</td>\n",
       "      <td>72</td>\n",
       "      <td>90</td>\n",
       "      <td>100</td>\n",
       "      <td>100</td>\n",
       "      <td>76</td>\n",
       "      <td>75</td>\n",
       "      <td>50.0</td>\n",
       "      <td>51</td>\n",
       "      <td>28</td>\n",
       "      <td>25</td>\n",
       "      <td>16</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>100</td>\n",
       "      <td>7</td>\n",
       "      <td>92</td>\n",
       "      <td>5</td>\n",
       "      <td>68</td>\n",
       "      <td>19</td>\n",
       "      <td>45</td>\n",
       "      <td>86</td>\n",
       "      <td>34</td>\n",
       "      <td>100.0</td>\n",
       "      <td>45</td>\n",
       "      <td>74</td>\n",
       "      <td>23</td>\n",
       "      <td>67</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>67</td>\n",
       "      <td>49</td>\n",
       "      <td>83</td>\n",
       "      <td>100</td>\n",
       "      <td>100</td>\n",
       "      <td>81</td>\n",
       "      <td>80</td>\n",
       "      <td>60</td>\n",
       "      <td>60</td>\n",
       "      <td>40.0</td>\n",
       "      <td>40</td>\n",
       "      <td>33</td>\n",
       "      <td>20</td>\n",
       "      <td>47</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   input1  input2  input3  input4  input5  input6  input7  input8  input9  \\\n",
       "0      47     100      27      81      57      37      26       0       0   \n",
       "1       0      89      27     100      42      75      29      45      15   \n",
       "2       0      57      31      68      72      90     100     100      76   \n",
       "3       0     100       7      92       5      68      19      45      86   \n",
       "4       0      67      49      83     100     100      81      80      60   \n",
       "\n",
       "   input10  input11  input12  input13  input14  input15  input16  class  \n",
       "0       23     56.0       53      100       90       40       98      8  \n",
       "1       15     37.0        0       69        2      100        6      2  \n",
       "2       75     50.0       51       28       25       16        0      1  \n",
       "3       34    100.0       45       74       23       67        0      4  \n",
       "4       60     40.0       40       33       20       47        0      1  "
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0        False\n",
       "1        False\n",
       "2        False\n",
       "3        False\n",
       "4        False\n",
       "         ...  \n",
       "10987    False\n",
       "10988    False\n",
       "10989    False\n",
       "10990    False\n",
       "10991    False\n",
       "Name: input2, Length: 10992, dtype: bool"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data['input2'] < 50"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>input1</th>\n",
       "      <th>input2</th>\n",
       "      <th>input3</th>\n",
       "      <th>input4</th>\n",
       "      <th>input5</th>\n",
       "      <th>input6</th>\n",
       "      <th>input7</th>\n",
       "      <th>input8</th>\n",
       "      <th>input9</th>\n",
       "      <th>input10</th>\n",
       "      <th>input11</th>\n",
       "      <th>input12</th>\n",
       "      <th>input13</th>\n",
       "      <th>input14</th>\n",
       "      <th>input15</th>\n",
       "      <th>input16</th>\n",
       "      <th>class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>0</td>\n",
       "      <td>39</td>\n",
       "      <td>2</td>\n",
       "      <td>62</td>\n",
       "      <td>11</td>\n",
       "      <td>5</td>\n",
       "      <td>63</td>\n",
       "      <td>0</td>\n",
       "      <td>100</td>\n",
       "      <td>43</td>\n",
       "      <td>89.0</td>\n",
       "      <td>99</td>\n",
       "      <td>36</td>\n",
       "      <td>100</td>\n",
       "      <td>0</td>\n",
       "      <td>57</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>0</td>\n",
       "      <td>46</td>\n",
       "      <td>49</td>\n",
       "      <td>64</td>\n",
       "      <td>78</td>\n",
       "      <td>87</td>\n",
       "      <td>100</td>\n",
       "      <td>100</td>\n",
       "      <td>91</td>\n",
       "      <td>75</td>\n",
       "      <td>85.0</td>\n",
       "      <td>49</td>\n",
       "      <td>75</td>\n",
       "      <td>24</td>\n",
       "      <td>89</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>31</td>\n",
       "      <td>15</td>\n",
       "      <td>63</td>\n",
       "      <td>30</td>\n",
       "      <td>88</td>\n",
       "      <td>52</td>\n",
       "      <td>100</td>\n",
       "      <td>79</td>\n",
       "      <td>82.0</td>\n",
       "      <td>100</td>\n",
       "      <td>56</td>\n",
       "      <td>82</td>\n",
       "      <td>79</td>\n",
       "      <td>64</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>137</th>\n",
       "      <td>0</td>\n",
       "      <td>42</td>\n",
       "      <td>30</td>\n",
       "      <td>56</td>\n",
       "      <td>59</td>\n",
       "      <td>72</td>\n",
       "      <td>86</td>\n",
       "      <td>94</td>\n",
       "      <td>100</td>\n",
       "      <td>100</td>\n",
       "      <td>80.0</td>\n",
       "      <td>68</td>\n",
       "      <td>62</td>\n",
       "      <td>35</td>\n",
       "      <td>53</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>146</th>\n",
       "      <td>0</td>\n",
       "      <td>42</td>\n",
       "      <td>34</td>\n",
       "      <td>57</td>\n",
       "      <td>64</td>\n",
       "      <td>79</td>\n",
       "      <td>95</td>\n",
       "      <td>100</td>\n",
       "      <td>100</td>\n",
       "      <td>85</td>\n",
       "      <td>81.0</td>\n",
       "      <td>57</td>\n",
       "      <td>62</td>\n",
       "      <td>28</td>\n",
       "      <td>45</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10805</th>\n",
       "      <td>0</td>\n",
       "      <td>43</td>\n",
       "      <td>32</td>\n",
       "      <td>58</td>\n",
       "      <td>67</td>\n",
       "      <td>79</td>\n",
       "      <td>100</td>\n",
       "      <td>100</td>\n",
       "      <td>89</td>\n",
       "      <td>76</td>\n",
       "      <td>76.0</td>\n",
       "      <td>51</td>\n",
       "      <td>63</td>\n",
       "      <td>25</td>\n",
       "      <td>59</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10866</th>\n",
       "      <td>0</td>\n",
       "      <td>42</td>\n",
       "      <td>40</td>\n",
       "      <td>60</td>\n",
       "      <td>74</td>\n",
       "      <td>81</td>\n",
       "      <td>100</td>\n",
       "      <td>100</td>\n",
       "      <td>90</td>\n",
       "      <td>74</td>\n",
       "      <td>78.0</td>\n",
       "      <td>48</td>\n",
       "      <td>66</td>\n",
       "      <td>23</td>\n",
       "      <td>65</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10879</th>\n",
       "      <td>0</td>\n",
       "      <td>45</td>\n",
       "      <td>42</td>\n",
       "      <td>70</td>\n",
       "      <td>66</td>\n",
       "      <td>100</td>\n",
       "      <td>60</td>\n",
       "      <td>62</td>\n",
       "      <td>47</td>\n",
       "      <td>25</td>\n",
       "      <td>1.0</td>\n",
       "      <td>5</td>\n",
       "      <td>43</td>\n",
       "      <td>0</td>\n",
       "      <td>100</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10880</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>29</td>\n",
       "      <td>24</td>\n",
       "      <td>57</td>\n",
       "      <td>50</td>\n",
       "      <td>83</td>\n",
       "      <td>77</td>\n",
       "      <td>100</td>\n",
       "      <td>100</td>\n",
       "      <td>80.0</td>\n",
       "      <td>69</td>\n",
       "      <td>64</td>\n",
       "      <td>35</td>\n",
       "      <td>51</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10933</th>\n",
       "      <td>0</td>\n",
       "      <td>15</td>\n",
       "      <td>29</td>\n",
       "      <td>40</td>\n",
       "      <td>59</td>\n",
       "      <td>68</td>\n",
       "      <td>85</td>\n",
       "      <td>97</td>\n",
       "      <td>100</td>\n",
       "      <td>100</td>\n",
       "      <td>79.0</td>\n",
       "      <td>67</td>\n",
       "      <td>60</td>\n",
       "      <td>34</td>\n",
       "      <td>46</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>379 rows × 17 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       input1  input2  input3  input4  input5  input6  input7  input8  input9  \\\n",
       "7           0      39       2      62      11       5      63       0     100   \n",
       "21          0      46      49      64      78      87     100     100      91   \n",
       "34          0       0      31      15      63      30      88      52     100   \n",
       "137         0      42      30      56      59      72      86      94     100   \n",
       "146         0      42      34      57      64      79      95     100     100   \n",
       "...       ...     ...     ...     ...     ...     ...     ...     ...     ...   \n",
       "10805       0      43      32      58      67      79     100     100      89   \n",
       "10866       0      42      40      60      74      81     100     100      90   \n",
       "10879       0      45      42      70      66     100      60      62      47   \n",
       "10880       0       3      29      24      57      50      83      77     100   \n",
       "10933       0      15      29      40      59      68      85      97     100   \n",
       "\n",
       "       input10  input11  input12  input13  input14  input15  input16  class  \n",
       "7           43     89.0       99       36      100        0       57      0  \n",
       "21          75     85.0       49       75       24       89        0      1  \n",
       "34          79     82.0      100       56       82       79       64      9  \n",
       "137        100     80.0       68       62       35       53        0      1  \n",
       "146         85     81.0       57       62       28       45        0      1  \n",
       "...        ...      ...      ...      ...      ...      ...      ...    ...  \n",
       "10805       76     76.0       51       63       25       59        0      1  \n",
       "10866       74     78.0       48       66       23       65        0      1  \n",
       "10879       25      1.0        5       43        0      100        4      1  \n",
       "10880      100     80.0       69       64       35       51        0      1  \n",
       "10933      100     79.0       67       60       34       46        0      1  \n",
       "\n",
       "[379 rows x 17 columns]"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[data['input2'] < 50]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>input1</th>\n",
       "      <th>input2</th>\n",
       "      <th>input3</th>\n",
       "      <th>input4</th>\n",
       "      <th>input5</th>\n",
       "      <th>input6</th>\n",
       "      <th>input7</th>\n",
       "      <th>input8</th>\n",
       "      <th>input9</th>\n",
       "      <th>input10</th>\n",
       "      <th>input11</th>\n",
       "      <th>input12</th>\n",
       "      <th>input13</th>\n",
       "      <th>input14</th>\n",
       "      <th>input15</th>\n",
       "      <th>input16</th>\n",
       "      <th>class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>89</td>\n",
       "      <td>27</td>\n",
       "      <td>100</td>\n",
       "      <td>42</td>\n",
       "      <td>75</td>\n",
       "      <td>29</td>\n",
       "      <td>45</td>\n",
       "      <td>15</td>\n",
       "      <td>15</td>\n",
       "      <td>37.0</td>\n",
       "      <td>0</td>\n",
       "      <td>69</td>\n",
       "      <td>2</td>\n",
       "      <td>100</td>\n",
       "      <td>6</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>57</td>\n",
       "      <td>31</td>\n",
       "      <td>68</td>\n",
       "      <td>72</td>\n",
       "      <td>90</td>\n",
       "      <td>100</td>\n",
       "      <td>100</td>\n",
       "      <td>76</td>\n",
       "      <td>75</td>\n",
       "      <td>50.0</td>\n",
       "      <td>51</td>\n",
       "      <td>28</td>\n",
       "      <td>25</td>\n",
       "      <td>16</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>100</td>\n",
       "      <td>7</td>\n",
       "      <td>92</td>\n",
       "      <td>5</td>\n",
       "      <td>68</td>\n",
       "      <td>19</td>\n",
       "      <td>45</td>\n",
       "      <td>86</td>\n",
       "      <td>34</td>\n",
       "      <td>100.0</td>\n",
       "      <td>45</td>\n",
       "      <td>74</td>\n",
       "      <td>23</td>\n",
       "      <td>67</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>67</td>\n",
       "      <td>49</td>\n",
       "      <td>83</td>\n",
       "      <td>100</td>\n",
       "      <td>100</td>\n",
       "      <td>81</td>\n",
       "      <td>80</td>\n",
       "      <td>60</td>\n",
       "      <td>60</td>\n",
       "      <td>40.0</td>\n",
       "      <td>40</td>\n",
       "      <td>33</td>\n",
       "      <td>20</td>\n",
       "      <td>47</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0</td>\n",
       "      <td>100</td>\n",
       "      <td>3</td>\n",
       "      <td>72</td>\n",
       "      <td>26</td>\n",
       "      <td>35</td>\n",
       "      <td>85</td>\n",
       "      <td>35</td>\n",
       "      <td>100</td>\n",
       "      <td>71</td>\n",
       "      <td>73.0</td>\n",
       "      <td>97</td>\n",
       "      <td>65</td>\n",
       "      <td>49</td>\n",
       "      <td>66</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10969</th>\n",
       "      <td>0</td>\n",
       "      <td>66</td>\n",
       "      <td>47</td>\n",
       "      <td>97</td>\n",
       "      <td>50</td>\n",
       "      <td>100</td>\n",
       "      <td>51</td>\n",
       "      <td>60</td>\n",
       "      <td>50</td>\n",
       "      <td>20</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>35</td>\n",
       "      <td>4</td>\n",
       "      <td>100</td>\n",
       "      <td>9</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10970</th>\n",
       "      <td>0</td>\n",
       "      <td>94</td>\n",
       "      <td>63</td>\n",
       "      <td>100</td>\n",
       "      <td>100</td>\n",
       "      <td>96</td>\n",
       "      <td>81</td>\n",
       "      <td>49</td>\n",
       "      <td>82</td>\n",
       "      <td>0</td>\n",
       "      <td>44.0</td>\n",
       "      <td>14</td>\n",
       "      <td>11</td>\n",
       "      <td>44</td>\n",
       "      <td>89</td>\n",
       "      <td>47</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10982</th>\n",
       "      <td>0</td>\n",
       "      <td>85</td>\n",
       "      <td>21</td>\n",
       "      <td>100</td>\n",
       "      <td>66</td>\n",
       "      <td>98</td>\n",
       "      <td>83</td>\n",
       "      <td>77</td>\n",
       "      <td>90</td>\n",
       "      <td>53</td>\n",
       "      <td>100.0</td>\n",
       "      <td>30</td>\n",
       "      <td>75</td>\n",
       "      <td>10</td>\n",
       "      <td>35</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10983</th>\n",
       "      <td>0</td>\n",
       "      <td>63</td>\n",
       "      <td>16</td>\n",
       "      <td>90</td>\n",
       "      <td>100</td>\n",
       "      <td>100</td>\n",
       "      <td>90</td>\n",
       "      <td>73</td>\n",
       "      <td>53</td>\n",
       "      <td>40</td>\n",
       "      <td>22.0</td>\n",
       "      <td>6</td>\n",
       "      <td>11</td>\n",
       "      <td>0</td>\n",
       "      <td>48</td>\n",
       "      <td>18</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10984</th>\n",
       "      <td>0</td>\n",
       "      <td>74</td>\n",
       "      <td>35</td>\n",
       "      <td>100</td>\n",
       "      <td>100</td>\n",
       "      <td>96</td>\n",
       "      <td>94</td>\n",
       "      <td>63</td>\n",
       "      <td>52</td>\n",
       "      <td>35</td>\n",
       "      <td>0.0</td>\n",
       "      <td>11</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>77</td>\n",
       "      <td>6</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>2364 rows × 17 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       input1  input2  input3  input4  input5  input6  input7  input8  input9  \\\n",
       "1           0      89      27     100      42      75      29      45      15   \n",
       "2           0      57      31      68      72      90     100     100      76   \n",
       "3           0     100       7      92       5      68      19      45      86   \n",
       "4           0      67      49      83     100     100      81      80      60   \n",
       "6           0     100       3      72      26      35      85      35     100   \n",
       "...       ...     ...     ...     ...     ...     ...     ...     ...     ...   \n",
       "10969       0      66      47      97      50     100      51      60      50   \n",
       "10970       0      94      63     100     100      96      81      49      82   \n",
       "10982       0      85      21     100      66      98      83      77      90   \n",
       "10983       0      63      16      90     100     100      90      73      53   \n",
       "10984       0      74      35     100     100      96      94      63      52   \n",
       "\n",
       "       input10  input11  input12  input13  input14  input15  input16  class  \n",
       "1           15     37.0        0       69        2      100        6      2  \n",
       "2           75     50.0       51       28       25       16        0      1  \n",
       "3           34    100.0       45       74       23       67        0      4  \n",
       "4           60     40.0       40       33       20       47        0      1  \n",
       "6           71     73.0       97       65       49       66        0      4  \n",
       "...        ...      ...      ...      ...      ...      ...      ...    ...  \n",
       "10969       20      1.0        0       35        4      100        9      1  \n",
       "10970        0     44.0       14       11       44       89       47      7  \n",
       "10982       53    100.0       30       75       10       35        0      3  \n",
       "10983       40     22.0        6       11        0       48       18      7  \n",
       "10984       35      0.0       11        5        0       77        6      2  \n",
       "\n",
       "[2364 rows x 17 columns]"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[data['input1'] == 0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(10992, 17)"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Missing values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "input1       0\n",
       "input2       0\n",
       "input3       0\n",
       "input4       0\n",
       "input5       0\n",
       "input6       0\n",
       "input7       0\n",
       "input8       0\n",
       "input9       0\n",
       "input10      0\n",
       "input11    248\n",
       "input12      0\n",
       "input13      0\n",
       "input14      0\n",
       "input15      0\n",
       "input16      0\n",
       "class        0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.isnull().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data.isnull().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>input1</th>\n",
       "      <th>input2</th>\n",
       "      <th>input3</th>\n",
       "      <th>input4</th>\n",
       "      <th>input5</th>\n",
       "      <th>input6</th>\n",
       "      <th>input7</th>\n",
       "      <th>input8</th>\n",
       "      <th>input9</th>\n",
       "      <th>input10</th>\n",
       "      <th>input11</th>\n",
       "      <th>input12</th>\n",
       "      <th>input13</th>\n",
       "      <th>input14</th>\n",
       "      <th>input15</th>\n",
       "      <th>input16</th>\n",
       "      <th>class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>58</th>\n",
       "      <td>18</td>\n",
       "      <td>73</td>\n",
       "      <td>0</td>\n",
       "      <td>73</td>\n",
       "      <td>10</td>\n",
       "      <td>24</td>\n",
       "      <td>54</td>\n",
       "      <td>0</td>\n",
       "      <td>100</td>\n",
       "      <td>22</td>\n",
       "      <td>NaN</td>\n",
       "      <td>71</td>\n",
       "      <td>52</td>\n",
       "      <td>100</td>\n",
       "      <td>5</td>\n",
       "      <td>84</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>87</th>\n",
       "      <td>47</td>\n",
       "      <td>93</td>\n",
       "      <td>1</td>\n",
       "      <td>67</td>\n",
       "      <td>0</td>\n",
       "      <td>16</td>\n",
       "      <td>53</td>\n",
       "      <td>0</td>\n",
       "      <td>100</td>\n",
       "      <td>32</td>\n",
       "      <td>NaN</td>\n",
       "      <td>81</td>\n",
       "      <td>42</td>\n",
       "      <td>100</td>\n",
       "      <td>7</td>\n",
       "      <td>62</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>122</th>\n",
       "      <td>45</td>\n",
       "      <td>100</td>\n",
       "      <td>10</td>\n",
       "      <td>67</td>\n",
       "      <td>7</td>\n",
       "      <td>16</td>\n",
       "      <td>56</td>\n",
       "      <td>0</td>\n",
       "      <td>100</td>\n",
       "      <td>31</td>\n",
       "      <td>NaN</td>\n",
       "      <td>79</td>\n",
       "      <td>38</td>\n",
       "      <td>95</td>\n",
       "      <td>0</td>\n",
       "      <td>61</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>164</th>\n",
       "      <td>21</td>\n",
       "      <td>75</td>\n",
       "      <td>40</td>\n",
       "      <td>100</td>\n",
       "      <td>85</td>\n",
       "      <td>94</td>\n",
       "      <td>67</td>\n",
       "      <td>63</td>\n",
       "      <td>100</td>\n",
       "      <td>43</td>\n",
       "      <td>NaN</td>\n",
       "      <td>11</td>\n",
       "      <td>47</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>193</th>\n",
       "      <td>30</td>\n",
       "      <td>100</td>\n",
       "      <td>4</td>\n",
       "      <td>67</td>\n",
       "      <td>0</td>\n",
       "      <td>24</td>\n",
       "      <td>30</td>\n",
       "      <td>0</td>\n",
       "      <td>66</td>\n",
       "      <td>11</td>\n",
       "      <td>NaN</td>\n",
       "      <td>41</td>\n",
       "      <td>100</td>\n",
       "      <td>82</td>\n",
       "      <td>64</td>\n",
       "      <td>93</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10717</th>\n",
       "      <td>56</td>\n",
       "      <td>100</td>\n",
       "      <td>38</td>\n",
       "      <td>69</td>\n",
       "      <td>0</td>\n",
       "      <td>41</td>\n",
       "      <td>44</td>\n",
       "      <td>38</td>\n",
       "      <td>100</td>\n",
       "      <td>40</td>\n",
       "      <td>NaN</td>\n",
       "      <td>68</td>\n",
       "      <td>80</td>\n",
       "      <td>34</td>\n",
       "      <td>74</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10870</th>\n",
       "      <td>80</td>\n",
       "      <td>100</td>\n",
       "      <td>100</td>\n",
       "      <td>97</td>\n",
       "      <td>48</td>\n",
       "      <td>68</td>\n",
       "      <td>7</td>\n",
       "      <td>38</td>\n",
       "      <td>0</td>\n",
       "      <td>7</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>98</td>\n",
       "      <td>22</td>\n",
       "      <td>9</td>\n",
       "      <td>12</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10889</th>\n",
       "      <td>47</td>\n",
       "      <td>100</td>\n",
       "      <td>35</td>\n",
       "      <td>95</td>\n",
       "      <td>0</td>\n",
       "      <td>66</td>\n",
       "      <td>31</td>\n",
       "      <td>43</td>\n",
       "      <td>100</td>\n",
       "      <td>48</td>\n",
       "      <td>NaN</td>\n",
       "      <td>66</td>\n",
       "      <td>82</td>\n",
       "      <td>34</td>\n",
       "      <td>79</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10908</th>\n",
       "      <td>100</td>\n",
       "      <td>72</td>\n",
       "      <td>77</td>\n",
       "      <td>100</td>\n",
       "      <td>20</td>\n",
       "      <td>74</td>\n",
       "      <td>71</td>\n",
       "      <td>56</td>\n",
       "      <td>94</td>\n",
       "      <td>81</td>\n",
       "      <td>NaN</td>\n",
       "      <td>39</td>\n",
       "      <td>65</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10947</th>\n",
       "      <td>6</td>\n",
       "      <td>83</td>\n",
       "      <td>99</td>\n",
       "      <td>86</td>\n",
       "      <td>80</td>\n",
       "      <td>100</td>\n",
       "      <td>0</td>\n",
       "      <td>80</td>\n",
       "      <td>100</td>\n",
       "      <td>81</td>\n",
       "      <td>NaN</td>\n",
       "      <td>57</td>\n",
       "      <td>71</td>\n",
       "      <td>28</td>\n",
       "      <td>54</td>\n",
       "      <td>0</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>248 rows × 17 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       input1  input2  input3  input4  input5  input6  input7  input8  input9  \\\n",
       "58         18      73       0      73      10      24      54       0     100   \n",
       "87         47      93       1      67       0      16      53       0     100   \n",
       "122        45     100      10      67       7      16      56       0     100   \n",
       "164        21      75      40     100      85      94      67      63     100   \n",
       "193        30     100       4      67       0      24      30       0      66   \n",
       "...       ...     ...     ...     ...     ...     ...     ...     ...     ...   \n",
       "10717      56     100      38      69       0      41      44      38     100   \n",
       "10870      80     100     100      97      48      68       7      38       0   \n",
       "10889      47     100      35      95       0      66      31      43     100   \n",
       "10908     100      72      77     100      20      74      71      56      94   \n",
       "10947       6      83      99      86      80     100       0      80     100   \n",
       "\n",
       "       input10  input11  input12  input13  input14  input15  input16  class  \n",
       "58          22      NaN       71       52      100        5       84      0  \n",
       "87          32      NaN       81       42      100        7       62      0  \n",
       "122         31      NaN       79       38       95        0       61      0  \n",
       "164         43      NaN       11       47        0        0       13      3  \n",
       "193         11      NaN       41      100       82       64       93      0  \n",
       "...        ...      ...      ...      ...      ...      ...      ...    ...  \n",
       "10717       40      NaN       68       80       34       74        0      4  \n",
       "10870        7      NaN        0       98       22        9       12      6  \n",
       "10889       48      NaN       66       82       34       79        0      4  \n",
       "10908       81      NaN       39       65        0        0        0      9  \n",
       "10947       81      NaN       57       71       28       54        0      9  \n",
       "\n",
       "[248 rows x 17 columns]"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[data['input11'].isnull()]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [],
   "source": [
    "data['input11']=data['input11'].fillna(0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>input1</th>\n",
       "      <th>input2</th>\n",
       "      <th>input3</th>\n",
       "      <th>input4</th>\n",
       "      <th>input5</th>\n",
       "      <th>input6</th>\n",
       "      <th>input7</th>\n",
       "      <th>input8</th>\n",
       "      <th>input9</th>\n",
       "      <th>input10</th>\n",
       "      <th>input11</th>\n",
       "      <th>input12</th>\n",
       "      <th>input13</th>\n",
       "      <th>input14</th>\n",
       "      <th>input15</th>\n",
       "      <th>input16</th>\n",
       "      <th>class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "Empty DataFrame\n",
       "Columns: [input1, input2, input3, input4, input5, input6, input7, input8, input9, input10, input11, input12, input13, input14, input15, input16, class]\n",
       "Index: []"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[data['input11'].isnull()]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [],
   "source": [
    "data=pd.read_csv('dataset_32_pendigits.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [],
   "source": [
    "someVari=data['input11']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0         56\n",
       "1         37\n",
       "2         50\n",
       "3        100\n",
       "4         40\n",
       "        ... \n",
       "10987     87\n",
       "10988     21\n",
       "10989    100\n",
       "10990    100\n",
       "10991    100\n",
       "Name: input11, Length: 10992, dtype: object"
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "someVari.fillna('mean')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Mathematical Operations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(38.814319505094616, 426647, 1173.5956715594484)"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data['input1'].mean(),data['input1'].sum(),data['input1'].var()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [],
   "source": [
    "sampDatasetOf1=pd.DataFrame(np.ones(10),columns=['column1'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>column1</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   column1\n",
       "0      1.0\n",
       "1      1.0\n",
       "2      1.0\n",
       "3      1.0\n",
       "4      1.0\n",
       "5      1.0\n",
       "6      1.0\n",
       "7      1.0\n",
       "8      1.0\n",
       "9      1.0"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sampDatasetOf1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>column1</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>6.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>7.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>8.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>9.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>10.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   column1\n",
       "0      1.0\n",
       "1      2.0\n",
       "2      3.0\n",
       "3      4.0\n",
       "4      5.0\n",
       "5      6.0\n",
       "6      7.0\n",
       "7      8.0\n",
       "8      9.0\n",
       "9     10.0"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.cumsum(sampDatasetOf1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     1.0\n",
       "1     2.0\n",
       "2     3.0\n",
       "3     4.0\n",
       "4     5.0\n",
       "5     6.0\n",
       "6     7.0\n",
       "7     8.0\n",
       "8     9.0\n",
       "9    10.0\n",
       "Name: column1, dtype: float64"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.cumsum(sampDatasetOf1['column1'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Map, Apply, Applymap  in Pandas"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### MAP  to map values or functions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It works on a series and supports functions, dictionary or series, and do the operation elementwise in the series"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [],
   "source": [
    "seriesObj=data['input1']\n",
    "pandasDFseriesObj=data[['input1']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0        47\n",
       "1         0\n",
       "2         0\n",
       "3         0\n",
       "4         0\n",
       "         ..\n",
       "10987    36\n",
       "10988    16\n",
       "10989    56\n",
       "10990    19\n",
       "10991    38\n",
       "Name: input1, Length: 10992, dtype: int64"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "seriesObj"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>input1</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>47</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10987</th>\n",
       "      <td>36</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10988</th>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10989</th>\n",
       "      <td>56</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10990</th>\n",
       "      <td>19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10991</th>\n",
       "      <td>38</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>10992 rows × 1 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       input1\n",
       "0          47\n",
       "1           0\n",
       "2           0\n",
       "3           0\n",
       "4           0\n",
       "...       ...\n",
       "10987      36\n",
       "10988      16\n",
       "10989      56\n",
       "10990      19\n",
       "10991      38\n",
       "\n",
       "[10992 rows x 1 columns]"
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pandasDFseriesObj"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0        47\n",
       "1         0\n",
       "2         0\n",
       "3         0\n",
       "4         0\n",
       "         ..\n",
       "10987    36\n",
       "10988    16\n",
       "10989    56\n",
       "10990    19\n",
       "10991    38\n",
       "Name: input1, Length: 10992, dtype: object"
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "seriesObj.map(str)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dtype('O')"
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "seriesObj.map(str).dtype"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'47'"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "seriesObj.map(str)[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [],
   "source": [
    "dictCheck={'a':10,'b':20,'c':30}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [],
   "source": [
    "sampNumpy=np.array(['a','b','c','a','b','c'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataFSamp=pd.DataFrame(sampNumpy,columns=['columnName'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>columnName</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>a</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>b</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>c</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>a</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>b</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>c</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  columnName\n",
       "0          a\n",
       "1          b\n",
       "2          c\n",
       "3          a\n",
       "4          b\n",
       "5          c"
      ]
     },
     "execution_count": 85,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataFSamp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    10\n",
       "1    20\n",
       "2    30\n",
       "3    10\n",
       "4    20\n",
       "5    30\n",
       "Name: columnName, dtype: int64"
      ]
     },
     "execution_count": 86,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataFSamp['columnName'].map(dictCheck)#function, dict, or Series"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [],
   "source": [
    "def checkFunct(x):\n",
    "    return dictCheck[x]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    10\n",
       "1    20\n",
       "2    30\n",
       "3    10\n",
       "4    20\n",
       "5    30\n",
       "Name: columnName, dtype: int64"
      ]
     },
     "execution_count": 88,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataFSamp['columnName'].map(checkFunct)#function, dict, or Series"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " The map function does not work on DataFrame, map works on each element of the series"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### APPLY"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As the name suggests, applies a function along any axis of the DataFrame. and also can be applied in series"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dtype('int64')"
      ]
     },
     "execution_count": 89,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "seriesObj.dtype"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "pandas.core.series.Series"
      ]
     },
     "execution_count": 90,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(seriesObj)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0        843.0\n",
       "1        653.0\n",
       "2        840.0\n",
       "3        769.0\n",
       "4        861.0\n",
       "         ...  \n",
       "10987    855.0\n",
       "10988    617.0\n",
       "10989    862.0\n",
       "10990    794.0\n",
       "10991    753.0\n",
       "Length: 10992, dtype: float64"
      ]
     },
     "execution_count": 92,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.apply(sum,axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "input1     426647.0\n",
       "input2     935642.0\n",
       "input3     446337.0\n",
       "input4     920846.0\n",
       "input5     547076.0\n",
       "input6     720780.0\n",
       "input7     563013.0\n",
       "input8     489133.0\n",
       "input9     625099.0\n",
       "input10    370386.0\n",
       "input11         NaN\n",
       "input12    382813.0\n",
       "input13    604805.0\n",
       "input14    384028.0\n",
       "input15    519784.0\n",
       "input16    317068.0\n",
       "class       48712.0\n",
       "dtype: float64"
      ]
     },
     "execution_count": 93,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.apply(sum)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "input1     426647.0\n",
       "input2     935642.0\n",
       "input3     446337.0\n",
       "input4     920846.0\n",
       "input5     547076.0\n",
       "input6     720780.0\n",
       "input7     563013.0\n",
       "input8     489133.0\n",
       "input9     625099.0\n",
       "input10    370386.0\n",
       "input11         NaN\n",
       "input12    382813.0\n",
       "input13    604805.0\n",
       "input14    384028.0\n",
       "input15    519784.0\n",
       "input16    317068.0\n",
       "class       48712.0\n",
       "dtype: float64"
      ]
     },
     "execution_count": 94,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.apply(sum,axis=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>columnName</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>a</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>b</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>c</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>a</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>b</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>c</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  columnName\n",
       "0          a\n",
       "1          b\n",
       "2          c\n",
       "3          a\n",
       "4          b\n",
       "5          c"
      ]
     },
     "execution_count": 95,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataFSamp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    10\n",
       "1    20\n",
       "2    30\n",
       "3    10\n",
       "4    20\n",
       "5    30\n",
       "Name: columnName, dtype: int64"
      ]
     },
     "execution_count": 96,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataFSamp['columnName'].apply(checkFunct)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### APPLY with LAMBDA "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "lambda function are a way to define functions withour actually defining them"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {},
   "outputs": [],
   "source": [
    "def checkFunc(x):\n",
    "    if x < 10:\n",
    "        return 'G10'\n",
    "    elif x < 50:\n",
    "        return 'G50'\n",
    "    elif x < 90:\n",
    "        return 'G90'\n",
    "    else:\n",
    "        return 'NO'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0         81\n",
       "1        100\n",
       "2         68\n",
       "3         92\n",
       "4         83\n",
       "        ... \n",
       "10987     70\n",
       "10988    100\n",
       "10989     79\n",
       "10990     61\n",
       "10991     81\n",
       "Name: input4, Length: 10992, dtype: int64"
      ]
     },
     "execution_count": 100,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data['input4']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0        G90\n",
       "1         NO\n",
       "2        G90\n",
       "3         NO\n",
       "4        G90\n",
       "        ... \n",
       "10987    G90\n",
       "10988     NO\n",
       "10989    G90\n",
       "10990    G90\n",
       "10991    G90\n",
       "Name: input4, Length: 10992, dtype: object"
      ]
     },
     "execution_count": 101,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data['input4'].apply(lambda x: checkFunc(x))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0        G90\n",
       "1         NO\n",
       "2        G90\n",
       "3         NO\n",
       "4        G90\n",
       "        ... \n",
       "10987    G90\n",
       "10988     NO\n",
       "10989    G90\n",
       "10990    G90\n",
       "10991    G90\n",
       "Name: input4, Length: 10992, dtype: object"
      ]
     },
     "execution_count": 102,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data['input4'].map(checkFunc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0        G90\n",
       "1         NO\n",
       "2        G90\n",
       "3         NO\n",
       "4        G90\n",
       "        ... \n",
       "10987    G90\n",
       "10988     NO\n",
       "10989    G90\n",
       "10990    G90\n",
       "10991    G90\n",
       "Name: input4, Length: 10992, dtype: object"
      ]
     },
     "execution_count": 103,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data['input4'].apply(checkFunc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0:00:00.002997\n"
     ]
    }
   ],
   "source": [
    "import datetime\n",
    "a=datetime.datetime.now()\n",
    "data['input4'].apply(checkFunc)\n",
    "b=datetime.datetime.now()\n",
    "print (b-a)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0:00:00.002999\n"
     ]
    }
   ],
   "source": [
    "import datetime\n",
    "a=datetime.datetime.now()\n",
    "data['input4'].map(checkFunc)\n",
    "b=datetime.datetime.now()\n",
    "print (b-a)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0:00:00.004999\n"
     ]
    }
   ],
   "source": [
    "import datetime\n",
    "a=datetime.datetime.now()\n",
    "data['input4'].apply(lambda x: checkFunc(x))\n",
    "b=datetime.datetime.now()\n",
    "print (b-a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ApplyMAP"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This method applies a function that accepts and returns a scalar\n",
    "to every element of a DataFrame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>input1</th>\n",
       "      <th>input2</th>\n",
       "      <th>input3</th>\n",
       "      <th>input4</th>\n",
       "      <th>input5</th>\n",
       "      <th>input6</th>\n",
       "      <th>input7</th>\n",
       "      <th>input8</th>\n",
       "      <th>input9</th>\n",
       "      <th>input10</th>\n",
       "      <th>input11</th>\n",
       "      <th>input12</th>\n",
       "      <th>input13</th>\n",
       "      <th>input14</th>\n",
       "      <th>input15</th>\n",
       "      <th>input16</th>\n",
       "      <th>class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>G50</td>\n",
       "      <td>NO</td>\n",
       "      <td>G50</td>\n",
       "      <td>G90</td>\n",
       "      <td>G90</td>\n",
       "      <td>G50</td>\n",
       "      <td>G50</td>\n",
       "      <td>G10</td>\n",
       "      <td>G10</td>\n",
       "      <td>G50</td>\n",
       "      <td>G90</td>\n",
       "      <td>G90</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>G50</td>\n",
       "      <td>NO</td>\n",
       "      <td>G10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>G10</td>\n",
       "      <td>G90</td>\n",
       "      <td>G50</td>\n",
       "      <td>NO</td>\n",
       "      <td>G50</td>\n",
       "      <td>G90</td>\n",
       "      <td>G50</td>\n",
       "      <td>G50</td>\n",
       "      <td>G50</td>\n",
       "      <td>G50</td>\n",
       "      <td>G50</td>\n",
       "      <td>G10</td>\n",
       "      <td>G90</td>\n",
       "      <td>G10</td>\n",
       "      <td>NO</td>\n",
       "      <td>G10</td>\n",
       "      <td>G10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>G10</td>\n",
       "      <td>G90</td>\n",
       "      <td>G50</td>\n",
       "      <td>G90</td>\n",
       "      <td>G90</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>G90</td>\n",
       "      <td>G90</td>\n",
       "      <td>G90</td>\n",
       "      <td>G90</td>\n",
       "      <td>G50</td>\n",
       "      <td>G50</td>\n",
       "      <td>G50</td>\n",
       "      <td>G10</td>\n",
       "      <td>G10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>G10</td>\n",
       "      <td>NO</td>\n",
       "      <td>G10</td>\n",
       "      <td>NO</td>\n",
       "      <td>G10</td>\n",
       "      <td>G90</td>\n",
       "      <td>G50</td>\n",
       "      <td>G50</td>\n",
       "      <td>G90</td>\n",
       "      <td>G50</td>\n",
       "      <td>NO</td>\n",
       "      <td>G50</td>\n",
       "      <td>G90</td>\n",
       "      <td>G50</td>\n",
       "      <td>G90</td>\n",
       "      <td>G10</td>\n",
       "      <td>G10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>G10</td>\n",
       "      <td>G90</td>\n",
       "      <td>G50</td>\n",
       "      <td>G90</td>\n",
       "      <td>NO</td>\n",
       "      <td>NO</td>\n",
       "      <td>G90</td>\n",
       "      <td>G90</td>\n",
       "      <td>G90</td>\n",
       "      <td>G90</td>\n",
       "      <td>G50</td>\n",
       "      <td>G50</td>\n",
       "      <td>G50</td>\n",
       "      <td>G50</td>\n",
       "      <td>G50</td>\n",
       "      <td>G10</td>\n",
       "      <td>G10</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  input1 input2 input3 input4 input5 input6 input7 input8 input9 input10  \\\n",
       "0    G50     NO    G50    G90    G90    G50    G50    G10    G10     G50   \n",
       "1    G10    G90    G50     NO    G50    G90    G50    G50    G50     G50   \n",
       "2    G10    G90    G50    G90    G90     NO     NO     NO    G90     G90   \n",
       "3    G10     NO    G10     NO    G10    G90    G50    G50    G90     G50   \n",
       "4    G10    G90    G50    G90     NO     NO    G90    G90    G90     G90   \n",
       "\n",
       "  input11 input12 input13 input14 input15 input16 class  \n",
       "0     G90     G90      NO      NO     G50      NO   G10  \n",
       "1     G50     G10     G90     G10      NO     G10   G10  \n",
       "2     G90     G90     G50     G50     G50     G10   G10  \n",
       "3      NO     G50     G90     G50     G90     G10   G10  \n",
       "4     G50     G50     G50     G50     G50     G10   G10  "
      ]
     },
     "execution_count": 116,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.applymap(checkFunc).head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Map is defined in series only\n",
    "- map accepts dicts, Series, or functions\n",
    "- map is elementwise for Series\n",
    "- Apply works in the series or dataframe either column or rows\n",
    "- apply accepts only functions\n",
    "- Apply with lambda each element over the series or columns\n",
    "\n",
    "- applymap is elementwise for DataFrames"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "metadata": {},
   "outputs": [],
   "source": [
    "def someFunc(x,y):\n",
    "    try:\n",
    "        return (x/y)\n",
    "    except:\n",
    "        return 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>input1</th>\n",
       "      <th>input2</th>\n",
       "      <th>input3</th>\n",
       "      <th>input4</th>\n",
       "      <th>input5</th>\n",
       "      <th>input6</th>\n",
       "      <th>input7</th>\n",
       "      <th>input8</th>\n",
       "      <th>input9</th>\n",
       "      <th>input10</th>\n",
       "      <th>input11</th>\n",
       "      <th>input12</th>\n",
       "      <th>input13</th>\n",
       "      <th>input14</th>\n",
       "      <th>input15</th>\n",
       "      <th>input16</th>\n",
       "      <th>class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>47</td>\n",
       "      <td>100</td>\n",
       "      <td>27</td>\n",
       "      <td>81</td>\n",
       "      <td>57</td>\n",
       "      <td>37</td>\n",
       "      <td>26</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>23</td>\n",
       "      <td>56.0</td>\n",
       "      <td>53</td>\n",
       "      <td>100</td>\n",
       "      <td>90</td>\n",
       "      <td>40</td>\n",
       "      <td>98</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>89</td>\n",
       "      <td>27</td>\n",
       "      <td>100</td>\n",
       "      <td>42</td>\n",
       "      <td>75</td>\n",
       "      <td>29</td>\n",
       "      <td>45</td>\n",
       "      <td>15</td>\n",
       "      <td>15</td>\n",
       "      <td>37.0</td>\n",
       "      <td>0</td>\n",
       "      <td>69</td>\n",
       "      <td>2</td>\n",
       "      <td>100</td>\n",
       "      <td>6</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>57</td>\n",
       "      <td>31</td>\n",
       "      <td>68</td>\n",
       "      <td>72</td>\n",
       "      <td>90</td>\n",
       "      <td>100</td>\n",
       "      <td>100</td>\n",
       "      <td>76</td>\n",
       "      <td>75</td>\n",
       "      <td>50.0</td>\n",
       "      <td>51</td>\n",
       "      <td>28</td>\n",
       "      <td>25</td>\n",
       "      <td>16</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>100</td>\n",
       "      <td>7</td>\n",
       "      <td>92</td>\n",
       "      <td>5</td>\n",
       "      <td>68</td>\n",
       "      <td>19</td>\n",
       "      <td>45</td>\n",
       "      <td>86</td>\n",
       "      <td>34</td>\n",
       "      <td>100.0</td>\n",
       "      <td>45</td>\n",
       "      <td>74</td>\n",
       "      <td>23</td>\n",
       "      <td>67</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>67</td>\n",
       "      <td>49</td>\n",
       "      <td>83</td>\n",
       "      <td>100</td>\n",
       "      <td>100</td>\n",
       "      <td>81</td>\n",
       "      <td>80</td>\n",
       "      <td>60</td>\n",
       "      <td>60</td>\n",
       "      <td>40.0</td>\n",
       "      <td>40</td>\n",
       "      <td>33</td>\n",
       "      <td>20</td>\n",
       "      <td>47</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   input1  input2  input3  input4  input5  input6  input7  input8  input9  \\\n",
       "0      47     100      27      81      57      37      26       0       0   \n",
       "1       0      89      27     100      42      75      29      45      15   \n",
       "2       0      57      31      68      72      90     100     100      76   \n",
       "3       0     100       7      92       5      68      19      45      86   \n",
       "4       0      67      49      83     100     100      81      80      60   \n",
       "\n",
       "   input10  input11  input12  input13  input14  input15  input16  class  \n",
       "0       23     56.0       53      100       90       40       98      8  \n",
       "1       15     37.0        0       69        2      100        6      2  \n",
       "2       75     50.0       51       28       25       16        0      1  \n",
       "3       34    100.0       45       74       23       67        0      4  \n",
       "4       60     40.0       40       33       20       47        0      1  "
      ]
     },
     "execution_count": 124,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0        0.333333\n",
       "1        0.270000\n",
       "2        0.455882\n",
       "3        0.076087\n",
       "4        0.590361\n",
       "           ...   \n",
       "10987    0.342857\n",
       "10988    0.410000\n",
       "10989    0.341772\n",
       "10990    0.000000\n",
       "10991    0.456790\n",
       "Length: 10992, dtype: float64"
      ]
     },
     "execution_count": 125,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[['input3','input4']].apply(lambda x: someFunc(*x),axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Merge(Join), Concat / Append in Pandas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "metadata": {},
   "outputs": [],
   "source": [
    "table1=pd.DataFrame(data={'column1':[1,2,3,4],'column2':[5,6,7,8]})\n",
    "table2=pd.DataFrame(data={'column1':[9,10,11,12],'column2':[13,14,15,16]})\n",
    "table3=pd.DataFrame(data={'column3':[1,1,2,2],'column4':[5,6,7,8]})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "append operation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>column1</th>\n",
       "      <th>column2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>9</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10</td>\n",
       "      <td>14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>11</td>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>12</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   column1  column2\n",
       "0        1        5\n",
       "1        2        6\n",
       "2        3        7\n",
       "3        4        8\n",
       "0        9       13\n",
       "1       10       14\n",
       "2       11       15\n",
       "3       12       16"
      ]
     },
     "execution_count": 127,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.concat([table1,table2])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>column1</th>\n",
       "      <th>column2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>9</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10</td>\n",
       "      <td>14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>11</td>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>12</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   column1  column2\n",
       "0        1        5\n",
       "1        2        6\n",
       "2        3        7\n",
       "3        4        8\n",
       "0        9       13\n",
       "1       10       14\n",
       "2       11       15\n",
       "3       12       16"
      ]
     },
     "execution_count": 128,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "table1.append(table2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>column1</th>\n",
       "      <th>column2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>9</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10</td>\n",
       "      <td>14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>11</td>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>12</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   column1  column2\n",
       "0        1        5\n",
       "1        2        6\n",
       "2        3        7\n",
       "3        4        8\n",
       "0        9       13\n",
       "1       10       14\n",
       "2       11       15\n",
       "3       12       16"
      ]
     },
     "execution_count": 129,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "table1.append(table2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 130,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\themachine\\.conda\\envs\\python36\\lib\\site-packages\\ipykernel_launcher.py:1: FutureWarning: Sorting because non-concatenation axis is not aligned. A future version\n",
      "of pandas will change to not sort by default.\n",
      "\n",
      "To accept the future behavior, pass 'sort=False'.\n",
      "\n",
      "To retain the current behavior and silence the warning, pass 'sort=True'.\n",
      "\n",
      "  \"\"\"Entry point for launching an IPython kernel.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>column1</th>\n",
       "      <th>column2</th>\n",
       "      <th>column3</th>\n",
       "      <th>column4</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4.0</td>\n",
       "      <td>8.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>6.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2.0</td>\n",
       "      <td>7.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2.0</td>\n",
       "      <td>8.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   column1  column2  column3  column4\n",
       "0      1.0      5.0      NaN      NaN\n",
       "1      2.0      6.0      NaN      NaN\n",
       "2      3.0      7.0      NaN      NaN\n",
       "3      4.0      8.0      NaN      NaN\n",
       "0      NaN      NaN      1.0      5.0\n",
       "1      NaN      NaN      1.0      6.0\n",
       "2      NaN      NaN      2.0      7.0\n",
       "3      NaN      NaN      2.0      8.0"
      ]
     },
     "execution_count": 130,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.concat([table1,table3])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>column1</th>\n",
       "      <th>column2</th>\n",
       "      <th>column3</th>\n",
       "      <th>column4</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>6</td>\n",
       "      <td>1</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>7</td>\n",
       "      <td>2</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>8</td>\n",
       "      <td>2</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   column1  column2  column3  column4\n",
       "0        1        5        1        5\n",
       "1        2        6        1        6\n",
       "2        3        7        2        7\n",
       "3        4        8        2        8"
      ]
     },
     "execution_count": 131,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.concat([table1,table3],axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Merge"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 132,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>column1</th>\n",
       "      <th>column2</th>\n",
       "      <th>column3</th>\n",
       "      <th>column4</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>1.0</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>1.0</td>\n",
       "      <td>6.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>6</td>\n",
       "      <td>2.0</td>\n",
       "      <td>7.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2</td>\n",
       "      <td>6</td>\n",
       "      <td>2.0</td>\n",
       "      <td>8.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>3</td>\n",
       "      <td>7</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>4</td>\n",
       "      <td>8</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   column1  column2  column3  column4\n",
       "0        1        5      1.0      5.0\n",
       "1        1        5      1.0      6.0\n",
       "2        2        6      2.0      7.0\n",
       "3        2        6      2.0      8.0\n",
       "4        3        7      NaN      NaN\n",
       "5        4        8      NaN      NaN"
      ]
     },
     "execution_count": 132,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.merge(table1,table3,left_on=['column1'],right_on=['column3'],how='left')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 133,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>column1</th>\n",
       "      <th>column2</th>\n",
       "      <th>column3</th>\n",
       "      <th>column4</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>6</td>\n",
       "      <td>2</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2</td>\n",
       "      <td>6</td>\n",
       "      <td>2</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   column1  column2  column3  column4\n",
       "0        1        5        1        5\n",
       "1        1        5        1        6\n",
       "2        2        6        2        7\n",
       "3        2        6        2        8"
      ]
     },
     "execution_count": 133,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.merge(table1,table3,left_on=['column1'],right_on=['column3'],how='inner')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Learning with example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 135,
   "metadata": {},
   "outputs": [],
   "source": [
    "salesData=pd.read_excel('Sample - Superstore.xls')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 136,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Row ID</th>\n",
       "      <th>Order ID</th>\n",
       "      <th>Order Date</th>\n",
       "      <th>Ship Date</th>\n",
       "      <th>Ship Mode</th>\n",
       "      <th>Customer ID</th>\n",
       "      <th>Customer Name</th>\n",
       "      <th>Segment</th>\n",
       "      <th>Country</th>\n",
       "      <th>City</th>\n",
       "      <th>...</th>\n",
       "      <th>Postal Code</th>\n",
       "      <th>Region</th>\n",
       "      <th>Product ID</th>\n",
       "      <th>Category</th>\n",
       "      <th>Sub-Category</th>\n",
       "      <th>Product Name</th>\n",
       "      <th>Sales</th>\n",
       "      <th>Quantity</th>\n",
       "      <th>Discount</th>\n",
       "      <th>Profit</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>CA-2016-152156</td>\n",
       "      <td>2016-11-08</td>\n",
       "      <td>2016-11-11</td>\n",
       "      <td>Second Class</td>\n",
       "      <td>CG-12520</td>\n",
       "      <td>Claire Gute</td>\n",
       "      <td>Consumer</td>\n",
       "      <td>United States</td>\n",
       "      <td>Henderson</td>\n",
       "      <td>...</td>\n",
       "      <td>42420</td>\n",
       "      <td>South</td>\n",
       "      <td>FUR-BO-10001798</td>\n",
       "      <td>Furniture</td>\n",
       "      <td>Bookcases</td>\n",
       "      <td>Bush Somerset Collection Bookcase</td>\n",
       "      <td>261.96</td>\n",
       "      <td>2</td>\n",
       "      <td>0.0</td>\n",
       "      <td>41.9136</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>CA-2016-152156</td>\n",
       "      <td>2016-11-08</td>\n",
       "      <td>2016-11-11</td>\n",
       "      <td>Second Class</td>\n",
       "      <td>CG-12520</td>\n",
       "      <td>Claire Gute</td>\n",
       "      <td>Consumer</td>\n",
       "      <td>United States</td>\n",
       "      <td>Henderson</td>\n",
       "      <td>...</td>\n",
       "      <td>42420</td>\n",
       "      <td>South</td>\n",
       "      <td>FUR-CH-10000454</td>\n",
       "      <td>Furniture</td>\n",
       "      <td>Chairs</td>\n",
       "      <td>Hon Deluxe Fabric Upholstered Stacking Chairs,...</td>\n",
       "      <td>731.94</td>\n",
       "      <td>3</td>\n",
       "      <td>0.0</td>\n",
       "      <td>219.5820</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>2 rows × 21 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   Row ID        Order ID Order Date  Ship Date     Ship Mode Customer ID  \\\n",
       "0       1  CA-2016-152156 2016-11-08 2016-11-11  Second Class    CG-12520   \n",
       "1       2  CA-2016-152156 2016-11-08 2016-11-11  Second Class    CG-12520   \n",
       "\n",
       "  Customer Name   Segment        Country       City  ... Postal Code  Region  \\\n",
       "0   Claire Gute  Consumer  United States  Henderson  ...       42420   South   \n",
       "1   Claire Gute  Consumer  United States  Henderson  ...       42420   South   \n",
       "\n",
       "        Product ID   Category Sub-Category  \\\n",
       "0  FUR-BO-10001798  Furniture    Bookcases   \n",
       "1  FUR-CH-10000454  Furniture       Chairs   \n",
       "\n",
       "                                        Product Name   Sales  Quantity  \\\n",
       "0                  Bush Somerset Collection Bookcase  261.96         2   \n",
       "1  Hon Deluxe Fabric Upholstered Stacking Chairs,...  731.94         3   \n",
       "\n",
       "   Discount    Profit  \n",
       "0       0.0   41.9136  \n",
       "1       0.0  219.5820  \n",
       "\n",
       "[2 rows x 21 columns]"
      ]
     },
     "execution_count": 136,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "salesData.head(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. Shape of the data\n",
    "2. Datatype of the data columns\n",
    "3. How many categorical data columns data has\n",
    "4. Find the unique items in the categorical data columns\n",
    "5. which Sub-Category has Standard Deviation for Sales and Profit one point more than the Mean of Sales and Profit?\n",
    "6. identify the Sub-Categories where Top 20 Customers that have Sales more than the other Sub-Categories in the same Region.\n",
    "7. identify the Regions where Top 20 Customers  overall Profit Ratio (Profit as % of Sales) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 138,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['Row ID', 'Order ID', 'Order Date', 'Ship Date', 'Ship Mode',\n",
       "       'Customer ID', 'Customer Name', 'Segment', 'Country', 'City', 'State',\n",
       "       'Postal Code', 'Region', 'Product ID', 'Category', 'Sub-Category',\n",
       "       'Product Name', 'Sales', 'Quantity', 'Discount', 'Profit'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 138,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "salesData.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 137,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(9994, 21)"
      ]
     },
     "execution_count": 137,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "salesData.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 139,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Row ID</th>\n",
       "      <th>Postal Code</th>\n",
       "      <th>Sales</th>\n",
       "      <th>Quantity</th>\n",
       "      <th>Discount</th>\n",
       "      <th>Profit</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>9994.000000</td>\n",
       "      <td>9994.000000</td>\n",
       "      <td>9994.000000</td>\n",
       "      <td>9994.000000</td>\n",
       "      <td>9994.000000</td>\n",
       "      <td>9994.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>4997.500000</td>\n",
       "      <td>55190.379428</td>\n",
       "      <td>229.858001</td>\n",
       "      <td>3.789574</td>\n",
       "      <td>0.156203</td>\n",
       "      <td>28.656896</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>2885.163629</td>\n",
       "      <td>32063.693350</td>\n",
       "      <td>623.245101</td>\n",
       "      <td>2.225110</td>\n",
       "      <td>0.206452</td>\n",
       "      <td>234.260108</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>1040.000000</td>\n",
       "      <td>0.444000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-6599.978000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>2499.250000</td>\n",
       "      <td>23223.000000</td>\n",
       "      <td>17.280000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.728750</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>4997.500000</td>\n",
       "      <td>56430.500000</td>\n",
       "      <td>54.490000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>0.200000</td>\n",
       "      <td>8.666500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>7495.750000</td>\n",
       "      <td>90008.000000</td>\n",
       "      <td>209.940000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>0.200000</td>\n",
       "      <td>29.364000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>9994.000000</td>\n",
       "      <td>99301.000000</td>\n",
       "      <td>22638.480000</td>\n",
       "      <td>14.000000</td>\n",
       "      <td>0.800000</td>\n",
       "      <td>8399.976000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            Row ID   Postal Code         Sales     Quantity     Discount  \\\n",
       "count  9994.000000   9994.000000   9994.000000  9994.000000  9994.000000   \n",
       "mean   4997.500000  55190.379428    229.858001     3.789574     0.156203   \n",
       "std    2885.163629  32063.693350    623.245101     2.225110     0.206452   \n",
       "min       1.000000   1040.000000      0.444000     1.000000     0.000000   \n",
       "25%    2499.250000  23223.000000     17.280000     2.000000     0.000000   \n",
       "50%    4997.500000  56430.500000     54.490000     3.000000     0.200000   \n",
       "75%    7495.750000  90008.000000    209.940000     5.000000     0.200000   \n",
       "max    9994.000000  99301.000000  22638.480000    14.000000     0.800000   \n",
       "\n",
       "            Profit  \n",
       "count  9994.000000  \n",
       "mean     28.656896  \n",
       "std     234.260108  \n",
       "min   -6599.978000  \n",
       "25%       1.728750  \n",
       "50%       8.666500  \n",
       "75%      29.364000  \n",
       "max    8399.976000  "
      ]
     },
     "execution_count": 139,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "salesData.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## How many unique products we got"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 140,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['FUR-BO-10000112', 'FUR-BO-10000330', 'FUR-BO-10000362', ...,\n",
       "       'TEC-PH-10004924', 'TEC-PH-10004959', 'TEC-PH-10004977'],\n",
       "      dtype=object)"
      ]
     },
     "execution_count": 140,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.unique(salesData['Product ID'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 141,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1862"
      ]
     },
     "execution_count": 141,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(pd.unique(salesData['Product ID']))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 142,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Row ID                    int64\n",
       "Order ID                 object\n",
       "Order Date       datetime64[ns]\n",
       "Ship Date        datetime64[ns]\n",
       "Ship Mode                object\n",
       "Customer ID              object\n",
       "Customer Name            object\n",
       "Segment                  object\n",
       "Country                  object\n",
       "City                     object\n",
       "State                    object\n",
       "Postal Code               int64\n",
       "Region                   object\n",
       "Product ID               object\n",
       "Category                 object\n",
       "Sub-Category             object\n",
       "Product Name             object\n",
       "Sales                   float64\n",
       "Quantity                  int64\n",
       "Discount                float64\n",
       "Profit                  float64\n",
       "dtype: object"
      ]
     },
     "execution_count": 142,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "salesData.dtypes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 143,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Row ID >>>> 9994\n",
      "Order ID >>>> 5009\n",
      "Order Date >>>> 1237\n",
      "Ship Date >>>> 1334\n",
      "Ship Mode >>>> 4\n",
      "Customer ID >>>> 793\n",
      "Customer Name >>>> 793\n",
      "Segment >>>> 3\n",
      "Country >>>> 1\n",
      "City >>>> 531\n",
      "State >>>> 49\n",
      "Postal Code >>>> 631\n",
      "Region >>>> 4\n",
      "Product ID >>>> 1862\n",
      "Category >>>> 3\n",
      "Sub-Category >>>> 17\n",
      "Product Name >>>> 1850\n",
      "Sales >>>> 6144\n",
      "Quantity >>>> 14\n",
      "Discount >>>> 12\n",
      "Profit >>>> 7545\n"
     ]
    }
   ],
   "source": [
    "for col in salesData.columns:\n",
    "    print (col,'>>>>',len(np.unique(salesData[col])))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "which Sub-Category has Standard Deviation for Sales and Profit one point more than the Mean of Sales and Profit?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 144,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Row ID</th>\n",
       "      <th>Order ID</th>\n",
       "      <th>Order Date</th>\n",
       "      <th>Ship Date</th>\n",
       "      <th>Ship Mode</th>\n",
       "      <th>Customer ID</th>\n",
       "      <th>Customer Name</th>\n",
       "      <th>Segment</th>\n",
       "      <th>Country</th>\n",
       "      <th>City</th>\n",
       "      <th>...</th>\n",
       "      <th>Postal Code</th>\n",
       "      <th>Region</th>\n",
       "      <th>Product ID</th>\n",
       "      <th>Category</th>\n",
       "      <th>Sub-Category</th>\n",
       "      <th>Product Name</th>\n",
       "      <th>Sales</th>\n",
       "      <th>Quantity</th>\n",
       "      <th>Discount</th>\n",
       "      <th>Profit</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>CA-2016-152156</td>\n",
       "      <td>2016-11-08</td>\n",
       "      <td>2016-11-11</td>\n",
       "      <td>Second Class</td>\n",
       "      <td>CG-12520</td>\n",
       "      <td>Claire Gute</td>\n",
       "      <td>Consumer</td>\n",
       "      <td>United States</td>\n",
       "      <td>Henderson</td>\n",
       "      <td>...</td>\n",
       "      <td>42420</td>\n",
       "      <td>South</td>\n",
       "      <td>FUR-BO-10001798</td>\n",
       "      <td>Furniture</td>\n",
       "      <td>Bookcases</td>\n",
       "      <td>Bush Somerset Collection Bookcase</td>\n",
       "      <td>261.96</td>\n",
       "      <td>2</td>\n",
       "      <td>0.0</td>\n",
       "      <td>41.9136</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1 rows × 21 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   Row ID        Order ID Order Date  Ship Date     Ship Mode Customer ID  \\\n",
       "0       1  CA-2016-152156 2016-11-08 2016-11-11  Second Class    CG-12520   \n",
       "\n",
       "  Customer Name   Segment        Country       City  ... Postal Code  Region  \\\n",
       "0   Claire Gute  Consumer  United States  Henderson  ...       42420   South   \n",
       "\n",
       "        Product ID   Category Sub-Category                       Product Name  \\\n",
       "0  FUR-BO-10001798  Furniture    Bookcases  Bush Somerset Collection Bookcase   \n",
       "\n",
       "    Sales  Quantity  Discount   Profit  \n",
       "0  261.96         2       0.0  41.9136  \n",
       "\n",
       "[1 rows x 21 columns]"
      ]
     },
     "execution_count": 144,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "salesData.head(1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 145,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['Row ID', 'Order ID', 'Order Date', 'Ship Date', 'Ship Mode',\n",
       "       'Customer ID', 'Customer Name', 'Segment', 'Country', 'City', 'State',\n",
       "       'Postal Code', 'Region', 'Product ID', 'Category', 'Sub-Category',\n",
       "       'Product Name', 'Sales', 'Quantity', 'Discount', 'Profit'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 145,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "salesData.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Pivot Tables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 165,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "savemeanInfo=pd.pivot_table(salesData,index='Sub-Category',values=['Sales','Profit'],aggfunc='mean').reset_index()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 166,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "saveSTDVarInfo=pd.pivot_table(salesData,index='Sub-Category',values=['Sales','Profit'],aggfunc='std').reset_index()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 167,
   "metadata": {},
   "outputs": [],
   "source": [
    "savemeanInfo.columns=['Sub-Category','MeanProfitVar','meanSalesVar']\n",
    "saveSTDVarInfo.columns=['Sub-Category','STDProfitVar','STDSalesVar']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 168,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Sub-Category</th>\n",
       "      <th>MeanProfitVar</th>\n",
       "      <th>meanSalesVar</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Accessories</td>\n",
       "      <td>54.111788</td>\n",
       "      <td>215.974604</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Appliances</td>\n",
       "      <td>38.922758</td>\n",
       "      <td>230.755710</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Art</td>\n",
       "      <td>8.200737</td>\n",
       "      <td>34.068834</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Binders</td>\n",
       "      <td>19.843574</td>\n",
       "      <td>133.560560</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Bookcases</td>\n",
       "      <td>-15.230509</td>\n",
       "      <td>503.859633</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Sub-Category  MeanProfitVar  meanSalesVar\n",
       "0  Accessories      54.111788    215.974604\n",
       "1   Appliances      38.922758    230.755710\n",
       "2          Art       8.200737     34.068834\n",
       "3      Binders      19.843574    133.560560\n",
       "4    Bookcases     -15.230509    503.859633"
      ]
     },
     "execution_count": 168,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "savemeanInfo.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 169,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Sub-Category</th>\n",
       "      <th>STDProfitVar</th>\n",
       "      <th>STDSalesVar</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Accessories</td>\n",
       "      <td>106.155455</td>\n",
       "      <td>334.965015</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Appliances</td>\n",
       "      <td>148.319146</td>\n",
       "      <td>388.949643</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Art</td>\n",
       "      <td>13.384264</td>\n",
       "      <td>60.122465</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Binders</td>\n",
       "      <td>306.196773</td>\n",
       "      <td>563.251188</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Bookcases</td>\n",
       "      <td>182.034224</td>\n",
       "      <td>638.748523</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Sub-Category  STDProfitVar  STDSalesVar\n",
       "0  Accessories    106.155455   334.965015\n",
       "1   Appliances    148.319146   388.949643\n",
       "2          Art     13.384264    60.122465\n",
       "3      Binders    306.196773   563.251188\n",
       "4    Bookcases    182.034224   638.748523"
      ]
     },
     "execution_count": 169,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "saveSTDVarInfo.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 171,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "savTheTable=pd.merge(savemeanInfo,saveSTDVarInfo,on='Sub-Category',how='left')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 178,
   "metadata": {},
   "outputs": [],
   "source": [
    "savTheTable['diffMean']=savTheTable['MeanProfitVar']-savTheTable['STDProfitVar']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 176,
   "metadata": {},
   "outputs": [],
   "source": [
    "savTheTable['diffSTD']=savTheTable['meanSalesVar']-savTheTable['STDSalesVar']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 179,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Sub-Category</th>\n",
       "      <th>MeanProfitVar</th>\n",
       "      <th>meanSalesVar</th>\n",
       "      <th>STDProfitVar</th>\n",
       "      <th>STDSalesVar</th>\n",
       "      <th>diffSTD</th>\n",
       "      <th>diffMean</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Accessories</td>\n",
       "      <td>54.111788</td>\n",
       "      <td>215.974604</td>\n",
       "      <td>106.155455</td>\n",
       "      <td>334.965015</td>\n",
       "      <td>-118.990411</td>\n",
       "      <td>-52.043667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Appliances</td>\n",
       "      <td>38.922758</td>\n",
       "      <td>230.755710</td>\n",
       "      <td>148.319146</td>\n",
       "      <td>388.949643</td>\n",
       "      <td>-158.193933</td>\n",
       "      <td>-109.396388</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Art</td>\n",
       "      <td>8.200737</td>\n",
       "      <td>34.068834</td>\n",
       "      <td>13.384264</td>\n",
       "      <td>60.122465</td>\n",
       "      <td>-26.053631</td>\n",
       "      <td>-5.183527</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Binders</td>\n",
       "      <td>19.843574</td>\n",
       "      <td>133.560560</td>\n",
       "      <td>306.196773</td>\n",
       "      <td>563.251188</td>\n",
       "      <td>-429.690627</td>\n",
       "      <td>-286.353199</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Bookcases</td>\n",
       "      <td>-15.230509</td>\n",
       "      <td>503.859633</td>\n",
       "      <td>182.034224</td>\n",
       "      <td>638.748523</td>\n",
       "      <td>-134.888890</td>\n",
       "      <td>-197.264733</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Chairs</td>\n",
       "      <td>43.095894</td>\n",
       "      <td>532.332420</td>\n",
       "      <td>127.099922</td>\n",
       "      <td>550.148243</td>\n",
       "      <td>-17.815823</td>\n",
       "      <td>-84.004028</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Copiers</td>\n",
       "      <td>817.909190</td>\n",
       "      <td>2198.941618</td>\n",
       "      <td>1460.921156</td>\n",
       "      <td>3175.665867</td>\n",
       "      <td>-976.724250</td>\n",
       "      <td>-643.011967</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Envelopes</td>\n",
       "      <td>27.418019</td>\n",
       "      <td>64.867724</td>\n",
       "      <td>35.354983</td>\n",
       "      <td>84.351633</td>\n",
       "      <td>-19.483908</td>\n",
       "      <td>-7.936964</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Fasteners</td>\n",
       "      <td>4.375660</td>\n",
       "      <td>13.936774</td>\n",
       "      <td>5.055053</td>\n",
       "      <td>12.416593</td>\n",
       "      <td>1.520181</td>\n",
       "      <td>-0.679393</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Furnishings</td>\n",
       "      <td>13.645918</td>\n",
       "      <td>95.825668</td>\n",
       "      <td>52.753228</td>\n",
       "      <td>147.893640</td>\n",
       "      <td>-52.067972</td>\n",
       "      <td>-39.107310</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Labels</td>\n",
       "      <td>15.236962</td>\n",
       "      <td>34.303055</td>\n",
       "      <td>34.084127</td>\n",
       "      <td>74.119287</td>\n",
       "      <td>-39.816232</td>\n",
       "      <td>-18.847165</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Machines</td>\n",
       "      <td>29.432669</td>\n",
       "      <td>1645.553313</td>\n",
       "      <td>1099.070067</td>\n",
       "      <td>2765.102088</td>\n",
       "      <td>-1119.548775</td>\n",
       "      <td>-1069.637398</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Paper</td>\n",
       "      <td>24.856620</td>\n",
       "      <td>57.284092</td>\n",
       "      <td>35.225214</td>\n",
       "      <td>78.167639</td>\n",
       "      <td>-20.883547</td>\n",
       "      <td>-10.368594</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Phones</td>\n",
       "      <td>50.073938</td>\n",
       "      <td>371.211534</td>\n",
       "      <td>113.251277</td>\n",
       "      <td>491.457343</td>\n",
       "      <td>-120.245808</td>\n",
       "      <td>-63.177339</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>Storage</td>\n",
       "      <td>25.152277</td>\n",
       "      <td>264.590553</td>\n",
       "      <td>81.923600</td>\n",
       "      <td>355.222507</td>\n",
       "      <td>-90.631953</td>\n",
       "      <td>-56.771323</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>Supplies</td>\n",
       "      <td>-6.258418</td>\n",
       "      <td>245.650200</td>\n",
       "      <td>105.059628</td>\n",
       "      <td>923.828753</td>\n",
       "      <td>-678.178553</td>\n",
       "      <td>-111.318047</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>Tables</td>\n",
       "      <td>-55.565771</td>\n",
       "      <td>648.794771</td>\n",
       "      <td>230.565006</td>\n",
       "      <td>615.774655</td>\n",
       "      <td>33.020116</td>\n",
       "      <td>-286.130778</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Sub-Category  MeanProfitVar  meanSalesVar  STDProfitVar  STDSalesVar  \\\n",
       "0   Accessories      54.111788    215.974604    106.155455   334.965015   \n",
       "1    Appliances      38.922758    230.755710    148.319146   388.949643   \n",
       "2           Art       8.200737     34.068834     13.384264    60.122465   \n",
       "3       Binders      19.843574    133.560560    306.196773   563.251188   \n",
       "4     Bookcases     -15.230509    503.859633    182.034224   638.748523   \n",
       "5        Chairs      43.095894    532.332420    127.099922   550.148243   \n",
       "6       Copiers     817.909190   2198.941618   1460.921156  3175.665867   \n",
       "7     Envelopes      27.418019     64.867724     35.354983    84.351633   \n",
       "8     Fasteners       4.375660     13.936774      5.055053    12.416593   \n",
       "9   Furnishings      13.645918     95.825668     52.753228   147.893640   \n",
       "10       Labels      15.236962     34.303055     34.084127    74.119287   \n",
       "11     Machines      29.432669   1645.553313   1099.070067  2765.102088   \n",
       "12        Paper      24.856620     57.284092     35.225214    78.167639   \n",
       "13       Phones      50.073938    371.211534    113.251277   491.457343   \n",
       "14      Storage      25.152277    264.590553     81.923600   355.222507   \n",
       "15     Supplies      -6.258418    245.650200    105.059628   923.828753   \n",
       "16       Tables     -55.565771    648.794771    230.565006   615.774655   \n",
       "\n",
       "        diffSTD     diffMean  \n",
       "0   -118.990411   -52.043667  \n",
       "1   -158.193933  -109.396388  \n",
       "2    -26.053631    -5.183527  \n",
       "3   -429.690627  -286.353199  \n",
       "4   -134.888890  -197.264733  \n",
       "5    -17.815823   -84.004028  \n",
       "6   -976.724250  -643.011967  \n",
       "7    -19.483908    -7.936964  \n",
       "8      1.520181    -0.679393  \n",
       "9    -52.067972   -39.107310  \n",
       "10   -39.816232   -18.847165  \n",
       "11 -1119.548775 -1069.637398  \n",
       "12   -20.883547   -10.368594  \n",
       "13  -120.245808   -63.177339  \n",
       "14   -90.631953   -56.771323  \n",
       "15  -678.178553  -111.318047  \n",
       "16    33.020116  -286.130778  "
      ]
     },
     "execution_count": 179,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "savTheTable#['MeanProfitVar']-savTheTable['STDProfitVar']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 183,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Sub-Category</th>\n",
       "      <th>MeanProfitVar</th>\n",
       "      <th>meanSalesVar</th>\n",
       "      <th>STDProfitVar</th>\n",
       "      <th>STDSalesVar</th>\n",
       "      <th>diffSTD</th>\n",
       "      <th>diffMean</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Fasteners</td>\n",
       "      <td>4.375660</td>\n",
       "      <td>13.936774</td>\n",
       "      <td>5.055053</td>\n",
       "      <td>12.416593</td>\n",
       "      <td>1.520181</td>\n",
       "      <td>-0.679393</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Art</td>\n",
       "      <td>8.200737</td>\n",
       "      <td>34.068834</td>\n",
       "      <td>13.384264</td>\n",
       "      <td>60.122465</td>\n",
       "      <td>-26.053631</td>\n",
       "      <td>-5.183527</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Envelopes</td>\n",
       "      <td>27.418019</td>\n",
       "      <td>64.867724</td>\n",
       "      <td>35.354983</td>\n",
       "      <td>84.351633</td>\n",
       "      <td>-19.483908</td>\n",
       "      <td>-7.936964</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Paper</td>\n",
       "      <td>24.856620</td>\n",
       "      <td>57.284092</td>\n",
       "      <td>35.225214</td>\n",
       "      <td>78.167639</td>\n",
       "      <td>-20.883547</td>\n",
       "      <td>-10.368594</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Labels</td>\n",
       "      <td>15.236962</td>\n",
       "      <td>34.303055</td>\n",
       "      <td>34.084127</td>\n",
       "      <td>74.119287</td>\n",
       "      <td>-39.816232</td>\n",
       "      <td>-18.847165</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Furnishings</td>\n",
       "      <td>13.645918</td>\n",
       "      <td>95.825668</td>\n",
       "      <td>52.753228</td>\n",
       "      <td>147.893640</td>\n",
       "      <td>-52.067972</td>\n",
       "      <td>-39.107310</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Accessories</td>\n",
       "      <td>54.111788</td>\n",
       "      <td>215.974604</td>\n",
       "      <td>106.155455</td>\n",
       "      <td>334.965015</td>\n",
       "      <td>-118.990411</td>\n",
       "      <td>-52.043667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>Storage</td>\n",
       "      <td>25.152277</td>\n",
       "      <td>264.590553</td>\n",
       "      <td>81.923600</td>\n",
       "      <td>355.222507</td>\n",
       "      <td>-90.631953</td>\n",
       "      <td>-56.771323</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Phones</td>\n",
       "      <td>50.073938</td>\n",
       "      <td>371.211534</td>\n",
       "      <td>113.251277</td>\n",
       "      <td>491.457343</td>\n",
       "      <td>-120.245808</td>\n",
       "      <td>-63.177339</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Chairs</td>\n",
       "      <td>43.095894</td>\n",
       "      <td>532.332420</td>\n",
       "      <td>127.099922</td>\n",
       "      <td>550.148243</td>\n",
       "      <td>-17.815823</td>\n",
       "      <td>-84.004028</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Appliances</td>\n",
       "      <td>38.922758</td>\n",
       "      <td>230.755710</td>\n",
       "      <td>148.319146</td>\n",
       "      <td>388.949643</td>\n",
       "      <td>-158.193933</td>\n",
       "      <td>-109.396388</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>Supplies</td>\n",
       "      <td>-6.258418</td>\n",
       "      <td>245.650200</td>\n",
       "      <td>105.059628</td>\n",
       "      <td>923.828753</td>\n",
       "      <td>-678.178553</td>\n",
       "      <td>-111.318047</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Bookcases</td>\n",
       "      <td>-15.230509</td>\n",
       "      <td>503.859633</td>\n",
       "      <td>182.034224</td>\n",
       "      <td>638.748523</td>\n",
       "      <td>-134.888890</td>\n",
       "      <td>-197.264733</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>Tables</td>\n",
       "      <td>-55.565771</td>\n",
       "      <td>648.794771</td>\n",
       "      <td>230.565006</td>\n",
       "      <td>615.774655</td>\n",
       "      <td>33.020116</td>\n",
       "      <td>-286.130778</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Binders</td>\n",
       "      <td>19.843574</td>\n",
       "      <td>133.560560</td>\n",
       "      <td>306.196773</td>\n",
       "      <td>563.251188</td>\n",
       "      <td>-429.690627</td>\n",
       "      <td>-286.353199</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Copiers</td>\n",
       "      <td>817.909190</td>\n",
       "      <td>2198.941618</td>\n",
       "      <td>1460.921156</td>\n",
       "      <td>3175.665867</td>\n",
       "      <td>-976.724250</td>\n",
       "      <td>-643.011967</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Machines</td>\n",
       "      <td>29.432669</td>\n",
       "      <td>1645.553313</td>\n",
       "      <td>1099.070067</td>\n",
       "      <td>2765.102088</td>\n",
       "      <td>-1119.548775</td>\n",
       "      <td>-1069.637398</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Sub-Category  MeanProfitVar  meanSalesVar  STDProfitVar  STDSalesVar  \\\n",
       "8     Fasteners       4.375660     13.936774      5.055053    12.416593   \n",
       "2           Art       8.200737     34.068834     13.384264    60.122465   \n",
       "7     Envelopes      27.418019     64.867724     35.354983    84.351633   \n",
       "12        Paper      24.856620     57.284092     35.225214    78.167639   \n",
       "10       Labels      15.236962     34.303055     34.084127    74.119287   \n",
       "9   Furnishings      13.645918     95.825668     52.753228   147.893640   \n",
       "0   Accessories      54.111788    215.974604    106.155455   334.965015   \n",
       "14      Storage      25.152277    264.590553     81.923600   355.222507   \n",
       "13       Phones      50.073938    371.211534    113.251277   491.457343   \n",
       "5        Chairs      43.095894    532.332420    127.099922   550.148243   \n",
       "1    Appliances      38.922758    230.755710    148.319146   388.949643   \n",
       "15     Supplies      -6.258418    245.650200    105.059628   923.828753   \n",
       "4     Bookcases     -15.230509    503.859633    182.034224   638.748523   \n",
       "16       Tables     -55.565771    648.794771    230.565006   615.774655   \n",
       "3       Binders      19.843574    133.560560    306.196773   563.251188   \n",
       "6       Copiers     817.909190   2198.941618   1460.921156  3175.665867   \n",
       "11     Machines      29.432669   1645.553313   1099.070067  2765.102088   \n",
       "\n",
       "        diffSTD     diffMean  \n",
       "8      1.520181    -0.679393  \n",
       "2    -26.053631    -5.183527  \n",
       "7    -19.483908    -7.936964  \n",
       "12   -20.883547   -10.368594  \n",
       "10   -39.816232   -18.847165  \n",
       "9    -52.067972   -39.107310  \n",
       "0   -118.990411   -52.043667  \n",
       "14   -90.631953   -56.771323  \n",
       "13  -120.245808   -63.177339  \n",
       "5    -17.815823   -84.004028  \n",
       "1   -158.193933  -109.396388  \n",
       "15  -678.178553  -111.318047  \n",
       "4   -134.888890  -197.264733  \n",
       "16    33.020116  -286.130778  \n",
       "3   -429.690627  -286.353199  \n",
       "6   -976.724250  -643.011967  \n",
       "11 -1119.548775 -1069.637398  "
      ]
     },
     "execution_count": 183,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "savTheTable.sort_values(by=['diffMean'],ascending=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 180,
   "metadata": {},
   "outputs": [],
   "source": [
    "# meanInfo=salesData.pivot_table(index='Sub-Category',values=['Sales','Profit'],aggfunc='mean').reset_index()\n",
    "# meanInfo.columns=['Sub-Category','MeanProfitVar','meanSalesVar']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 181,
   "metadata": {},
   "outputs": [],
   "source": [
    "# varInfo=salesData.pivot_table(index='Sub-Category',values=['Sales','Profit'],aggfunc='std').reset_index()\n",
    "# varInfo.columns=['Sub-Category','ProfitVar','SalesVar']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mergedInfo=pd.merge(meanInfo,varInfo,on=['Sub-Category'],how='left')\n",
    "mergedInfo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mergedInfo['diffVar']=mergedInfo['MeanProfitVar']-mergedInfo['ProfitVar']\n",
    "mergedInfo['diffMean']=mergedInfo['meanSalesVar']-mergedInfo['SalesVar']\n",
    "mergedInfo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "mergedInfo.sort_values(by='diffMean')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mergedInfo.sort_values(by='diffMean',ascending=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## identify the Sub-Categories where Top 20 Customers that have Sales more than the other Sub-Categories in the same Region."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 191,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Row ID</th>\n",
       "      <th>Order ID</th>\n",
       "      <th>Order Date</th>\n",
       "      <th>Ship Date</th>\n",
       "      <th>Ship Mode</th>\n",
       "      <th>Customer ID</th>\n",
       "      <th>Customer Name</th>\n",
       "      <th>Segment</th>\n",
       "      <th>Country</th>\n",
       "      <th>City</th>\n",
       "      <th>...</th>\n",
       "      <th>Postal Code</th>\n",
       "      <th>Region</th>\n",
       "      <th>Product ID</th>\n",
       "      <th>Category</th>\n",
       "      <th>Sub-Category</th>\n",
       "      <th>Product Name</th>\n",
       "      <th>Sales</th>\n",
       "      <th>Quantity</th>\n",
       "      <th>Discount</th>\n",
       "      <th>Profit</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>CA-2016-152156</td>\n",
       "      <td>2016-11-08</td>\n",
       "      <td>2016-11-11</td>\n",
       "      <td>Second Class</td>\n",
       "      <td>CG-12520</td>\n",
       "      <td>Claire Gute</td>\n",
       "      <td>Consumer</td>\n",
       "      <td>United States</td>\n",
       "      <td>Henderson</td>\n",
       "      <td>...</td>\n",
       "      <td>42420</td>\n",
       "      <td>South</td>\n",
       "      <td>FUR-BO-10001798</td>\n",
       "      <td>Furniture</td>\n",
       "      <td>Bookcases</td>\n",
       "      <td>Bush Somerset Collection Bookcase</td>\n",
       "      <td>261.96</td>\n",
       "      <td>2</td>\n",
       "      <td>0.0</td>\n",
       "      <td>41.9136</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1 rows × 21 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   Row ID        Order ID Order Date  Ship Date     Ship Mode Customer ID  \\\n",
       "0       1  CA-2016-152156 2016-11-08 2016-11-11  Second Class    CG-12520   \n",
       "\n",
       "  Customer Name   Segment        Country       City  ... Postal Code  Region  \\\n",
       "0   Claire Gute  Consumer  United States  Henderson  ...       42420   South   \n",
       "\n",
       "        Product ID   Category Sub-Category                       Product Name  \\\n",
       "0  FUR-BO-10001798  Furniture    Bookcases  Bush Somerset Collection Bookcase   \n",
       "\n",
       "    Sales  Quantity  Discount   Profit  \n",
       "0  261.96         2       0.0  41.9136  \n",
       "\n",
       "[1 rows x 21 columns]"
      ]
     },
     "execution_count": 191,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "salesData.head(1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 200,
   "metadata": {},
   "outputs": [],
   "source": [
    "customerName=pd.pivot_table(salesData,index=['Customer ID'],values=['Sales'],aggfunc='sum')\\\n",
    "        .reset_index().sort_values(by='Sales',ascending=False).head(20)['Customer ID'].values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 205,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['South', 'West', 'Central', 'East'], dtype=object)"
      ]
     },
     "execution_count": 205,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.unique(salesData['Region'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 209,
   "metadata": {},
   "outputs": [],
   "source": [
    "filtData=salesData[(salesData['Customer ID'].isin(customerName)) & (salesData['Region']=='South')]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 213,
   "metadata": {},
   "outputs": [],
   "source": [
    "filtData=salesData[(salesData['Customer ID'].isin(customerName))]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 218,
   "metadata": {},
   "outputs": [],
   "source": [
    "savThePivT=pd.pivot_table(filtData,index='Sub-Category',columns=['Region'],values=['Sales'])#.sort_values(by='Sales')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 220,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "MultiIndex([('Sales', 'Central'),\n",
       "            ('Sales',    'East'),\n",
       "            ('Sales',   'South'),\n",
       "            ('Sales',    'West')],\n",
       "           names=[None, 'Region'])"
      ]
     },
     "execution_count": 220,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "savThePivT.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 221,
   "metadata": {},
   "outputs": [],
   "source": [
    "savThePivT.columns=[''.join(i) for i in savThePivT.columns]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 222,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>SalesCentral</th>\n",
       "      <th>SalesEast</th>\n",
       "      <th>SalesSouth</th>\n",
       "      <th>SalesWest</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Sub-Category</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Accessories</th>\n",
       "      <td>36.027000</td>\n",
       "      <td>100.938889</td>\n",
       "      <td>498.000000</td>\n",
       "      <td>125.175667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Appliances</th>\n",
       "      <td>122.191000</td>\n",
       "      <td>222.697200</td>\n",
       "      <td>NaN</td>\n",
       "      <td>207.082500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Art</th>\n",
       "      <td>26.999111</td>\n",
       "      <td>49.739091</td>\n",
       "      <td>11.260750</td>\n",
       "      <td>39.927200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Binders</th>\n",
       "      <td>1486.914267</td>\n",
       "      <td>274.165625</td>\n",
       "      <td>427.466167</td>\n",
       "      <td>41.815200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Bookcases</th>\n",
       "      <td>1023.332000</td>\n",
       "      <td>995.506800</td>\n",
       "      <td>145.740000</td>\n",
       "      <td>413.488500</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              SalesCentral   SalesEast  SalesSouth   SalesWest\n",
       "Sub-Category                                                  \n",
       "Accessories      36.027000  100.938889  498.000000  125.175667\n",
       "Appliances      122.191000  222.697200         NaN  207.082500\n",
       "Art              26.999111   49.739091   11.260750   39.927200\n",
       "Binders        1486.914267  274.165625  427.466167   41.815200\n",
       "Bookcases      1023.332000  995.506800  145.740000  413.488500"
      ]
     },
     "execution_count": 222,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "savThePivT.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 223,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>SalesCentral</th>\n",
       "      <th>SalesEast</th>\n",
       "      <th>SalesSouth</th>\n",
       "      <th>SalesWest</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Sub-Category</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Labels</th>\n",
       "      <td>11.960000</td>\n",
       "      <td>44.400000</td>\n",
       "      <td>15.750000</td>\n",
       "      <td>21.170000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Fasteners</th>\n",
       "      <td>15.692000</td>\n",
       "      <td>4.360000</td>\n",
       "      <td>25.028000</td>\n",
       "      <td>7.280000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Art</th>\n",
       "      <td>26.999111</td>\n",
       "      <td>49.739091</td>\n",
       "      <td>11.260750</td>\n",
       "      <td>39.927200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Envelopes</th>\n",
       "      <td>35.093333</td>\n",
       "      <td>58.206000</td>\n",
       "      <td>23.340000</td>\n",
       "      <td>92.630000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Accessories</th>\n",
       "      <td>36.027000</td>\n",
       "      <td>100.938889</td>\n",
       "      <td>498.000000</td>\n",
       "      <td>125.175667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Paper</th>\n",
       "      <td>39.236286</td>\n",
       "      <td>41.751500</td>\n",
       "      <td>17.522800</td>\n",
       "      <td>88.103125</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Appliances</th>\n",
       "      <td>122.191000</td>\n",
       "      <td>222.697200</td>\n",
       "      <td>NaN</td>\n",
       "      <td>207.082500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Furnishings</th>\n",
       "      <td>129.247500</td>\n",
       "      <td>88.888182</td>\n",
       "      <td>156.064286</td>\n",
       "      <td>102.971250</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Storage</th>\n",
       "      <td>189.164500</td>\n",
       "      <td>447.360000</td>\n",
       "      <td>193.770000</td>\n",
       "      <td>199.485400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Supplies</th>\n",
       "      <td>197.585000</td>\n",
       "      <td>18.153000</td>\n",
       "      <td>3.328000</td>\n",
       "      <td>2983.951200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Tables</th>\n",
       "      <td>481.096250</td>\n",
       "      <td>199.836000</td>\n",
       "      <td>693.101000</td>\n",
       "      <td>647.938500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Phones</th>\n",
       "      <td>515.872444</td>\n",
       "      <td>452.395571</td>\n",
       "      <td>417.548000</td>\n",
       "      <td>948.608800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Chairs</th>\n",
       "      <td>1016.881667</td>\n",
       "      <td>787.300846</td>\n",
       "      <td>1667.744000</td>\n",
       "      <td>576.258667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Bookcases</th>\n",
       "      <td>1023.332000</td>\n",
       "      <td>995.506800</td>\n",
       "      <td>145.740000</td>\n",
       "      <td>413.488500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Binders</th>\n",
       "      <td>1486.914267</td>\n",
       "      <td>274.165625</td>\n",
       "      <td>427.466167</td>\n",
       "      <td>41.815200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Machines</th>\n",
       "      <td>4223.931000</td>\n",
       "      <td>4919.940000</td>\n",
       "      <td>10736.576667</td>\n",
       "      <td>1553.480000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Copiers</th>\n",
       "      <td>10149.955000</td>\n",
       "      <td>7674.971000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>8399.960000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              SalesCentral    SalesEast    SalesSouth    SalesWest\n",
       "Sub-Category                                                      \n",
       "Labels           11.960000    44.400000     15.750000    21.170000\n",
       "Fasteners        15.692000     4.360000     25.028000     7.280000\n",
       "Art              26.999111    49.739091     11.260750    39.927200\n",
       "Envelopes        35.093333    58.206000     23.340000    92.630000\n",
       "Accessories      36.027000   100.938889    498.000000   125.175667\n",
       "Paper            39.236286    41.751500     17.522800    88.103125\n",
       "Appliances      122.191000   222.697200           NaN   207.082500\n",
       "Furnishings     129.247500    88.888182    156.064286   102.971250\n",
       "Storage         189.164500   447.360000    193.770000   199.485400\n",
       "Supplies        197.585000    18.153000      3.328000  2983.951200\n",
       "Tables          481.096250   199.836000    693.101000   647.938500\n",
       "Phones          515.872444   452.395571    417.548000   948.608800\n",
       "Chairs         1016.881667   787.300846   1667.744000   576.258667\n",
       "Bookcases      1023.332000   995.506800    145.740000   413.488500\n",
       "Binders        1486.914267   274.165625    427.466167    41.815200\n",
       "Machines       4223.931000  4919.940000  10736.576667  1553.480000\n",
       "Copiers       10149.955000  7674.971000           NaN  8399.960000"
      ]
     },
     "execution_count": 223,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "savThePivT.sort_values(by='SalesCentral')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 216,
   "metadata": {},
   "outputs": [
    {
     "ename": "ValueError",
     "evalue": "The column label 'Sales' is not unique.\nFor a multi-index, the label must be a tuple with elements corresponding to each level.",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-216-fecd598a938c>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0mpd\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mpivot_table\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mfiltData\u001b[0m\u001b[1;33m,\u001b[0m\u001b[0mindex\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;34m'Sub-Category'\u001b[0m\u001b[1;33m,\u001b[0m\u001b[0mcolumns\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;34m'Region'\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m,\u001b[0m\u001b[0mvalues\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;34m'Sales'\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0msort_values\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mby\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;34m'Sales'\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[1;32mc:\\users\\themachine\\.conda\\envs\\python36\\lib\\site-packages\\pandas\\core\\frame.py\u001b[0m in \u001b[0;36msort_values\u001b[1;34m(self, by, axis, ascending, inplace, kind, na_position)\u001b[0m\n\u001b[0;32m   5006\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   5007\u001b[0m             \u001b[0mby\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mby\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;36m0\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 5008\u001b[1;33m             \u001b[0mk\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_get_label_or_level_values\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mby\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0maxis\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0maxis\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m   5009\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   5010\u001b[0m             \u001b[1;32mif\u001b[0m \u001b[0misinstance\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mascending\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m(\u001b[0m\u001b[0mtuple\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mlist\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mc:\\users\\themachine\\.conda\\envs\\python36\\lib\\site-packages\\pandas\\core\\generic.py\u001b[0m in \u001b[0;36m_get_label_or_level_values\u001b[1;34m(self, key, axis)\u001b[0m\n\u001b[0;32m   1795\u001b[0m                     \u001b[0mkey\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mkey\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   1796\u001b[0m                     \u001b[0mlabel_axis_name\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mlabel_axis_name\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 1797\u001b[1;33m                     \u001b[0mmulti_message\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mmulti_message\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m   1798\u001b[0m                 )\n\u001b[0;32m   1799\u001b[0m             )\n",
      "\u001b[1;31mValueError\u001b[0m: The column label 'Sales' is not unique.\nFor a multi-index, the label must be a tuple with elements corresponding to each level."
     ]
    }
   ],
   "source": [
    "pd.pivot_table(filtData,index='Sub-Category',columns=['Region'],values=['Sales']).sort_values(by='Sales')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 184,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Row ID</th>\n",
       "      <th>Order ID</th>\n",
       "      <th>Order Date</th>\n",
       "      <th>Ship Date</th>\n",
       "      <th>Ship Mode</th>\n",
       "      <th>Customer ID</th>\n",
       "      <th>Customer Name</th>\n",
       "      <th>Segment</th>\n",
       "      <th>Country</th>\n",
       "      <th>City</th>\n",
       "      <th>...</th>\n",
       "      <th>Postal Code</th>\n",
       "      <th>Region</th>\n",
       "      <th>Product ID</th>\n",
       "      <th>Category</th>\n",
       "      <th>Sub-Category</th>\n",
       "      <th>Product Name</th>\n",
       "      <th>Sales</th>\n",
       "      <th>Quantity</th>\n",
       "      <th>Discount</th>\n",
       "      <th>Profit</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>CA-2016-152156</td>\n",
       "      <td>2016-11-08</td>\n",
       "      <td>2016-11-11</td>\n",
       "      <td>Second Class</td>\n",
       "      <td>CG-12520</td>\n",
       "      <td>Claire Gute</td>\n",
       "      <td>Consumer</td>\n",
       "      <td>United States</td>\n",
       "      <td>Henderson</td>\n",
       "      <td>...</td>\n",
       "      <td>42420</td>\n",
       "      <td>South</td>\n",
       "      <td>FUR-BO-10001798</td>\n",
       "      <td>Furniture</td>\n",
       "      <td>Bookcases</td>\n",
       "      <td>Bush Somerset Collection Bookcase</td>\n",
       "      <td>261.96</td>\n",
       "      <td>2</td>\n",
       "      <td>0.0</td>\n",
       "      <td>41.9136</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1 rows × 21 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   Row ID        Order ID Order Date  Ship Date     Ship Mode Customer ID  \\\n",
       "0       1  CA-2016-152156 2016-11-08 2016-11-11  Second Class    CG-12520   \n",
       "\n",
       "  Customer Name   Segment        Country       City  ... Postal Code  Region  \\\n",
       "0   Claire Gute  Consumer  United States  Henderson  ...       42420   South   \n",
       "\n",
       "        Product ID   Category Sub-Category                       Product Name  \\\n",
       "0  FUR-BO-10001798  Furniture    Bookcases  Bush Somerset Collection Bookcase   \n",
       "\n",
       "    Sales  Quantity  Discount   Profit  \n",
       "0  261.96         2       0.0  41.9136  \n",
       "\n",
       "[1 rows x 21 columns]"
      ]
     },
     "execution_count": 184,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "salesData.head(1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 185,
   "metadata": {},
   "outputs": [],
   "source": [
    "regionSalesCustData=salesData.pivot_table(index=['Region','Customer ID'],values='Sales',aggfunc='sum').reset_index()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 186,
   "metadata": {},
   "outputs": [],
   "source": [
    "listOfCustomers=regionSalesCustData[regionSalesCustData['Region']=='Central'].sort_values(by='Sales',ascending=False).head(20)['Customer ID'].values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 187,
   "metadata": {},
   "outputs": [],
   "source": [
    "filtData=salesData[(salesData['Customer ID'].isin(listOfCustomers)) & (salesData['Region'] == 'Central') ]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 188,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "pivFiltData=filtData.pivot_table(index='Customer ID', columns=['Sub-Category'],values='Sales',aggfunc='sum')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 189,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Sub-Category\n",
       "Binders        33153.2340\n",
       "Copiers        25349.8300\n",
       "Machines       15035.7810\n",
       "Chairs         13541.4140\n",
       "Phones         10379.3520\n",
       "Supplies        8116.0820\n",
       "Appliances      5839.7030\n",
       "Accessories     4115.5540\n",
       "Storage         3680.5500\n",
       "Bookcases       1994.7788\n",
       "Paper           1770.5920\n",
       "Furnishings     1069.8600\n",
       "Labels           590.4240\n",
       "Tables           435.1650\n",
       "Envelopes        422.1280\n",
       "Art              352.6500\n",
       "Fasteners          7.0240\n",
       "dtype: float64"
      ]
     },
     "execution_count": 189,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pivFiltData.sum(axis=0).sort_values(ascending=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### identify the Regions where Top 20 Customers  overall Profit Ratio (Profit as % of Sales) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Group by"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A groupby operation involves some combination of splitting the object, applying a function, and combining the results. This can be used to group large amounts of data and compute operations on these groups."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 226,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Profit</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Region</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Central</th>\n",
       "      <td>39706.3625</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>East</th>\n",
       "      <td>91522.7800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>South</th>\n",
       "      <td>46749.4303</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>West</th>\n",
       "      <td>108418.4489</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              Profit\n",
       "Region              \n",
       "Central   39706.3625\n",
       "East      91522.7800\n",
       "South     46749.4303\n",
       "West     108418.4489"
      ]
     },
     "execution_count": 226,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "salesData.groupby('Region').sum()[['Profit']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 227,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>Row ID</th>\n",
       "      <th>Postal Code</th>\n",
       "      <th>Sales</th>\n",
       "      <th>Quantity</th>\n",
       "      <th>Discount</th>\n",
       "      <th>Profit</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Region</th>\n",
       "      <th>Customer ID</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">Central</th>\n",
       "      <th>AA-10315</th>\n",
       "      <td>23123</td>\n",
       "      <td>425470</td>\n",
       "      <td>4780.552</td>\n",
       "      <td>19</td>\n",
       "      <td>0.8</td>\n",
       "      <td>-650.5971</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>AA-10375</th>\n",
       "      <td>1617</td>\n",
       "      <td>136208</td>\n",
       "      <td>178.370</td>\n",
       "      <td>5</td>\n",
       "      <td>0.0</td>\n",
       "      <td>62.0658</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>AA-10480</th>\n",
       "      <td>54075</td>\n",
       "      <td>570205</td>\n",
       "      <td>1747.500</td>\n",
       "      <td>31</td>\n",
       "      <td>0.0</td>\n",
       "      <td>420.4986</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>AA-10645</th>\n",
       "      <td>24027</td>\n",
       "      <td>219360</td>\n",
       "      <td>1971.460</td>\n",
       "      <td>12</td>\n",
       "      <td>0.0</td>\n",
       "      <td>439.3138</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>AB-10015</th>\n",
       "      <td>25828</td>\n",
       "      <td>222257</td>\n",
       "      <td>576.564</td>\n",
       "      <td>6</td>\n",
       "      <td>0.2</td>\n",
       "      <td>114.3432</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                     Row ID  Postal Code     Sales  Quantity  Discount  \\\n",
       "Region  Customer ID                                                      \n",
       "Central AA-10315      23123       425470  4780.552        19       0.8   \n",
       "        AA-10375       1617       136208   178.370         5       0.0   \n",
       "        AA-10480      54075       570205  1747.500        31       0.0   \n",
       "        AA-10645      24027       219360  1971.460        12       0.0   \n",
       "        AB-10015      25828       222257   576.564         6       0.2   \n",
       "\n",
       "                       Profit  \n",
       "Region  Customer ID            \n",
       "Central AA-10315    -650.5971  \n",
       "        AA-10375      62.0658  \n",
       "        AA-10480     420.4986  \n",
       "        AA-10645     439.3138  \n",
       "        AB-10015     114.3432  "
      ]
     },
     "execution_count": 227,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "salesData.groupby(['Region','Customer ID']).sum().head(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 231,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Region</th>\n",
       "      <th>Customer ID</th>\n",
       "      <th>Sales</th>\n",
       "      <th>Profit</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Central</td>\n",
       "      <td>AA-10315</td>\n",
       "      <td>4780.552</td>\n",
       "      <td>333.653062</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Central</td>\n",
       "      <td>AA-10375</td>\n",
       "      <td>178.370</td>\n",
       "      <td>28.223319</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Central</td>\n",
       "      <td>AA-10480</td>\n",
       "      <td>1747.500</td>\n",
       "      <td>52.929506</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Central</td>\n",
       "      <td>AA-10645</td>\n",
       "      <td>1971.460</td>\n",
       "      <td>206.878156</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Central</td>\n",
       "      <td>AB-10015</td>\n",
       "      <td>576.564</td>\n",
       "      <td>35.390674</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2496</th>\n",
       "      <td>West</td>\n",
       "      <td>XP-21865</td>\n",
       "      <td>660.190</td>\n",
       "      <td>42.429516</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2497</th>\n",
       "      <td>West</td>\n",
       "      <td>YC-21895</td>\n",
       "      <td>1215.676</td>\n",
       "      <td>260.164030</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2498</th>\n",
       "      <td>West</td>\n",
       "      <td>YS-21880</td>\n",
       "      <td>5754.172</td>\n",
       "      <td>345.634555</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2499</th>\n",
       "      <td>West</td>\n",
       "      <td>ZC-21910</td>\n",
       "      <td>2641.089</td>\n",
       "      <td>85.120354</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2500</th>\n",
       "      <td>West</td>\n",
       "      <td>ZD-21925</td>\n",
       "      <td>306.200</td>\n",
       "      <td>6.864926</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>2501 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       Region Customer ID     Sales      Profit\n",
       "0     Central    AA-10315  4780.552  333.653062\n",
       "1     Central    AA-10375   178.370   28.223319\n",
       "2     Central    AA-10480  1747.500   52.929506\n",
       "3     Central    AA-10645  1971.460  206.878156\n",
       "4     Central    AB-10015   576.564   35.390674\n",
       "...       ...         ...       ...         ...\n",
       "2496     West    XP-21865   660.190   42.429516\n",
       "2497     West    YC-21895  1215.676  260.164030\n",
       "2498     West    YS-21880  5754.172  345.634555\n",
       "2499     West    ZC-21910  2641.089   85.120354\n",
       "2500     West    ZD-21925   306.200    6.864926\n",
       "\n",
       "[2501 rows x 4 columns]"
      ]
     },
     "execution_count": 231,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "salesData.groupby(['Region','Customer ID']).agg({'Sales':'var','Sales':'sum','Profit':'std'}).reset_index()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 232,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\themachine\\.conda\\envs\\python36\\lib\\site-packages\\pandas\\core\\groupby\\generic.py:1455: FutureWarning: using a dict with renaming is deprecated and will be removed\n",
      "in a future version.\n",
      "\n",
      "For column-specific groupby renaming, use named aggregation\n",
      "\n",
      "    >>> df.groupby(...).agg(name=('column', aggfunc))\n",
      "\n",
      "  return super().aggregate(arg, *args, **kwargs)\n"
     ]
    }
   ],
   "source": [
    "saveToEdit=salesData.groupby(['Region','Customer ID']).agg({'Sales':{'samValSales':'sum','varvalSales':'var'},'Profit':{'profitSum':'sum'}}).reset_index()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 233,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr th {\n",
       "        text-align: left;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>Region</th>\n",
       "      <th>Customer ID</th>\n",
       "      <th colspan=\"2\" halign=\"left\">Sales</th>\n",
       "      <th>Profit</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>samValSales</th>\n",
       "      <th>varvalSales</th>\n",
       "      <th>profitSum</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Central</td>\n",
       "      <td>AA-10315</td>\n",
       "      <td>4780.552</td>\n",
       "      <td>2.391345e+06</td>\n",
       "      <td>-650.5971</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Central</td>\n",
       "      <td>AA-10375</td>\n",
       "      <td>178.370</td>\n",
       "      <td>7.389632e+03</td>\n",
       "      <td>62.0658</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Central</td>\n",
       "      <td>AA-10480</td>\n",
       "      <td>1747.500</td>\n",
       "      <td>3.360151e+04</td>\n",
       "      <td>420.4986</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Central</td>\n",
       "      <td>AA-10645</td>\n",
       "      <td>1971.460</td>\n",
       "      <td>3.807022e+05</td>\n",
       "      <td>439.3138</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Central</td>\n",
       "      <td>AB-10015</td>\n",
       "      <td>576.564</td>\n",
       "      <td>2.778122e+04</td>\n",
       "      <td>114.3432</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2496</th>\n",
       "      <td>West</td>\n",
       "      <td>XP-21865</td>\n",
       "      <td>660.190</td>\n",
       "      <td>7.591865e+03</td>\n",
       "      <td>298.3414</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2497</th>\n",
       "      <td>West</td>\n",
       "      <td>YC-21895</td>\n",
       "      <td>1215.676</td>\n",
       "      <td>7.012548e+05</td>\n",
       "      <td>382.0575</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2498</th>\n",
       "      <td>West</td>\n",
       "      <td>YS-21880</td>\n",
       "      <td>5754.172</td>\n",
       "      <td>8.956381e+05</td>\n",
       "      <td>1551.5945</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2499</th>\n",
       "      <td>West</td>\n",
       "      <td>ZC-21910</td>\n",
       "      <td>2641.089</td>\n",
       "      <td>1.233752e+05</td>\n",
       "      <td>346.0942</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2500</th>\n",
       "      <td>West</td>\n",
       "      <td>ZD-21925</td>\n",
       "      <td>306.200</td>\n",
       "      <td>8.832815e+03</td>\n",
       "      <td>42.4662</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>2501 rows × 5 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       Region Customer ID       Sales                   Profit\n",
       "                          samValSales   varvalSales  profitSum\n",
       "0     Central    AA-10315    4780.552  2.391345e+06  -650.5971\n",
       "1     Central    AA-10375     178.370  7.389632e+03    62.0658\n",
       "2     Central    AA-10480    1747.500  3.360151e+04   420.4986\n",
       "3     Central    AA-10645    1971.460  3.807022e+05   439.3138\n",
       "4     Central    AB-10015     576.564  2.778122e+04   114.3432\n",
       "...       ...         ...         ...           ...        ...\n",
       "2496     West    XP-21865     660.190  7.591865e+03   298.3414\n",
       "2497     West    YC-21895    1215.676  7.012548e+05   382.0575\n",
       "2498     West    YS-21880    5754.172  8.956381e+05  1551.5945\n",
       "2499     West    ZC-21910    2641.089  1.233752e+05   346.0942\n",
       "2500     West    ZD-21925     306.200  8.832815e+03    42.4662\n",
       "\n",
       "[2501 rows x 5 columns]"
      ]
     },
     "execution_count": 233,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "saveToEdit"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 234,
   "metadata": {},
   "outputs": [],
   "source": [
    "saveToEdit.columns=['_'.join(i) for i in saveToEdit.columns]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 235,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Region_</th>\n",
       "      <th>Customer ID_</th>\n",
       "      <th>Sales_samValSales</th>\n",
       "      <th>Sales_varvalSales</th>\n",
       "      <th>Profit_profitSum</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Central</td>\n",
       "      <td>AA-10315</td>\n",
       "      <td>4780.552</td>\n",
       "      <td>2.391345e+06</td>\n",
       "      <td>-650.5971</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Central</td>\n",
       "      <td>AA-10375</td>\n",
       "      <td>178.370</td>\n",
       "      <td>7.389632e+03</td>\n",
       "      <td>62.0658</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Central</td>\n",
       "      <td>AA-10480</td>\n",
       "      <td>1747.500</td>\n",
       "      <td>3.360151e+04</td>\n",
       "      <td>420.4986</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Central</td>\n",
       "      <td>AA-10645</td>\n",
       "      <td>1971.460</td>\n",
       "      <td>3.807022e+05</td>\n",
       "      <td>439.3138</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Central</td>\n",
       "      <td>AB-10015</td>\n",
       "      <td>576.564</td>\n",
       "      <td>2.778122e+04</td>\n",
       "      <td>114.3432</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Region_ Customer ID_  Sales_samValSales  Sales_varvalSales  \\\n",
       "0  Central     AA-10315           4780.552       2.391345e+06   \n",
       "1  Central     AA-10375            178.370       7.389632e+03   \n",
       "2  Central     AA-10480           1747.500       3.360151e+04   \n",
       "3  Central     AA-10645           1971.460       3.807022e+05   \n",
       "4  Central     AB-10015            576.564       2.778122e+04   \n",
       "\n",
       "   Profit_profitSum  \n",
       "0         -650.5971  \n",
       "1           62.0658  \n",
       "2          420.4986  \n",
       "3          439.3138  \n",
       "4          114.3432  "
      ]
     },
     "execution_count": 235,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "saveToEdit.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 236,
   "metadata": {},
   "outputs": [],
   "source": [
    "saveForExample=salesData.groupby(['Region','Customer ID']).agg({'Sales':'sum','Profit':'sum'}).reset_index()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 237,
   "metadata": {},
   "outputs": [],
   "source": [
    "saveForExample['profitRatio']=saveForExample['Profit']/saveForExample['Sales']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 238,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "saveForExample=saveForExample.sort_values(['Region','profitRatio'],ascending=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 239,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "saveForExample=saveForExample.reset_index()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 240,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>index</th>\n",
       "      <th>Region</th>\n",
       "      <th>Customer ID</th>\n",
       "      <th>Sales</th>\n",
       "      <th>Profit</th>\n",
       "      <th>profitRatio</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2206</td>\n",
       "      <td>West</td>\n",
       "      <td>LB-16735</td>\n",
       "      <td>21.480</td>\n",
       "      <td>10.7400</td>\n",
       "      <td>0.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2367</td>\n",
       "      <td>West</td>\n",
       "      <td>RD-19810</td>\n",
       "      <td>146.820</td>\n",
       "      <td>73.4100</td>\n",
       "      <td>0.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2422</td>\n",
       "      <td>West</td>\n",
       "      <td>SM-20320</td>\n",
       "      <td>9.960</td>\n",
       "      <td>4.8804</td>\n",
       "      <td>0.490000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1939</td>\n",
       "      <td>West</td>\n",
       "      <td>CG-12040</td>\n",
       "      <td>11.760</td>\n",
       "      <td>5.7624</td>\n",
       "      <td>0.490000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2468</td>\n",
       "      <td>West</td>\n",
       "      <td>TP-21415</td>\n",
       "      <td>12.960</td>\n",
       "      <td>6.3504</td>\n",
       "      <td>0.490000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2496</th>\n",
       "      <td>340</td>\n",
       "      <td>Central</td>\n",
       "      <td>KN-16450</td>\n",
       "      <td>48.632</td>\n",
       "      <td>-121.5800</td>\n",
       "      <td>-2.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2497</th>\n",
       "      <td>550</td>\n",
       "      <td>Central</td>\n",
       "      <td>SJ-20500</td>\n",
       "      <td>97.264</td>\n",
       "      <td>-243.1600</td>\n",
       "      <td>-2.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2498</th>\n",
       "      <td>561</td>\n",
       "      <td>Central</td>\n",
       "      <td>SP-20920</td>\n",
       "      <td>23.992</td>\n",
       "      <td>-62.3792</td>\n",
       "      <td>-2.600000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2499</th>\n",
       "      <td>390</td>\n",
       "      <td>Central</td>\n",
       "      <td>MG-17695</td>\n",
       "      <td>145.504</td>\n",
       "      <td>-392.8595</td>\n",
       "      <td>-2.699991</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2500</th>\n",
       "      <td>195</td>\n",
       "      <td>Central</td>\n",
       "      <td>Dl-13600</td>\n",
       "      <td>1.624</td>\n",
       "      <td>-4.4660</td>\n",
       "      <td>-2.750000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>2501 rows × 6 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      index   Region Customer ID    Sales    Profit  profitRatio\n",
       "0      2206     West    LB-16735   21.480   10.7400     0.500000\n",
       "1      2367     West    RD-19810  146.820   73.4100     0.500000\n",
       "2      2422     West    SM-20320    9.960    4.8804     0.490000\n",
       "3      1939     West    CG-12040   11.760    5.7624     0.490000\n",
       "4      2468     West    TP-21415   12.960    6.3504     0.490000\n",
       "...     ...      ...         ...      ...       ...          ...\n",
       "2496    340  Central    KN-16450   48.632 -121.5800    -2.500000\n",
       "2497    550  Central    SJ-20500   97.264 -243.1600    -2.500000\n",
       "2498    561  Central    SP-20920   23.992  -62.3792    -2.600000\n",
       "2499    390  Central    MG-17695  145.504 -392.8595    -2.699991\n",
       "2500    195  Central    Dl-13600    1.624   -4.4660    -2.750000\n",
       "\n",
       "[2501 rows x 6 columns]"
      ]
     },
     "execution_count": 240,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "saveForExample"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 242,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>index</th>\n",
       "      <th>Region</th>\n",
       "      <th>Customer ID</th>\n",
       "      <th>Sales</th>\n",
       "      <th>Profit</th>\n",
       "      <th>profitRatio</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1127</th>\n",
       "      <td>1754</td>\n",
       "      <td>South</td>\n",
       "      <td>SM-20320</td>\n",
       "      <td>23669.196</td>\n",
       "      <td>-1787.0435</td>\n",
       "      <td>-0.075501</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1885</th>\n",
       "      <td>588</td>\n",
       "      <td>Central</td>\n",
       "      <td>TC-20980</td>\n",
       "      <td>18437.138</td>\n",
       "      <td>8745.0635</td>\n",
       "      <td>0.474318</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>2354</td>\n",
       "      <td>West</td>\n",
       "      <td>RB-19360</td>\n",
       "      <td>14345.276</td>\n",
       "      <td>6807.0879</td>\n",
       "      <td>0.474518</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1334</th>\n",
       "      <td>1250</td>\n",
       "      <td>East</td>\n",
       "      <td>TA-21385</td>\n",
       "      <td>13723.498</td>\n",
       "      <td>4599.2073</td>\n",
       "      <td>0.335134</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1909</th>\n",
       "      <td>6</td>\n",
       "      <td>Central</td>\n",
       "      <td>AB-10105</td>\n",
       "      <td>12181.594</td>\n",
       "      <td>5362.6135</td>\n",
       "      <td>0.440223</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      index   Region Customer ID      Sales     Profit  profitRatio\n",
       "1127   1754    South    SM-20320  23669.196 -1787.0435    -0.075501\n",
       "1885    588  Central    TC-20980  18437.138  8745.0635     0.474318\n",
       "18     2354     West    RB-19360  14345.276  6807.0879     0.474518\n",
       "1334   1250     East    TA-21385  13723.498  4599.2073     0.335134\n",
       "1909      6  Central    AB-10105  12181.594  5362.6135     0.440223"
      ]
     },
     "execution_count": 242,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "saveForExample.nlargest(5,'Sales')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 243,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr:last-of-type th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>profitRatio</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>&lt;lambda&gt;</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Region</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Central</th>\n",
       "      <td>0.479799</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>East</th>\n",
       "      <td>0.484697</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>South</th>\n",
       "      <td>0.481304</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>West</th>\n",
       "      <td>0.482885</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        profitRatio\n",
       "           <lambda>\n",
       "Region             \n",
       "Central    0.479799\n",
       "East       0.484697\n",
       "South      0.481304\n",
       "West       0.482885"
      ]
     },
     "execution_count": 243,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "saveForExample.groupby(['Region']).agg({'profitRatio':{lambda x: x.nlargest(20).mean()}})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 244,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.48288526990598823"
      ]
     },
     "execution_count": 244,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "saveForExample[saveForExample['Region']=='West'].head(20)['profitRatio'].mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Stacking"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### reorganized from being side by side on a horizontal position (the columns of the dataframe) to being stacked vertically on top of each other"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 245,
   "metadata": {},
   "outputs": [],
   "source": [
    "sampStackData=saveForExample.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 246,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>index</th>\n",
       "      <th>Region</th>\n",
       "      <th>Customer ID</th>\n",
       "      <th>Sales</th>\n",
       "      <th>Profit</th>\n",
       "      <th>profitRatio</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2206</td>\n",
       "      <td>West</td>\n",
       "      <td>LB-16735</td>\n",
       "      <td>21.48</td>\n",
       "      <td>10.7400</td>\n",
       "      <td>0.50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2367</td>\n",
       "      <td>West</td>\n",
       "      <td>RD-19810</td>\n",
       "      <td>146.82</td>\n",
       "      <td>73.4100</td>\n",
       "      <td>0.50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2422</td>\n",
       "      <td>West</td>\n",
       "      <td>SM-20320</td>\n",
       "      <td>9.96</td>\n",
       "      <td>4.8804</td>\n",
       "      <td>0.49</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1939</td>\n",
       "      <td>West</td>\n",
       "      <td>CG-12040</td>\n",
       "      <td>11.76</td>\n",
       "      <td>5.7624</td>\n",
       "      <td>0.49</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2468</td>\n",
       "      <td>West</td>\n",
       "      <td>TP-21415</td>\n",
       "      <td>12.96</td>\n",
       "      <td>6.3504</td>\n",
       "      <td>0.49</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   index Region Customer ID   Sales   Profit  profitRatio\n",
       "0   2206   West    LB-16735   21.48  10.7400         0.50\n",
       "1   2367   West    RD-19810  146.82  73.4100         0.50\n",
       "2   2422   West    SM-20320    9.96   4.8804         0.49\n",
       "3   1939   West    CG-12040   11.76   5.7624         0.49\n",
       "4   2468   West    TP-21415   12.96   6.3504         0.49"
      ]
     },
     "execution_count": 246,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sampStackData"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 249,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0  index              2206\n",
       "   Region             West\n",
       "   Customer ID    LB-16735\n",
       "   Sales             21.48\n",
       "   Profit            10.74\n",
       "   profitRatio         0.5\n",
       "1  index              2367\n",
       "   Region             West\n",
       "   Customer ID    RD-19810\n",
       "   Sales            146.82\n",
       "   Profit            73.41\n",
       "   profitRatio         0.5\n",
       "2  index              2422\n",
       "   Region             West\n",
       "   Customer ID    SM-20320\n",
       "   Sales              9.96\n",
       "   Profit           4.8804\n",
       "   profitRatio        0.49\n",
       "3  index              1939\n",
       "   Region             West\n",
       "   Customer ID    CG-12040\n",
       "   Sales             11.76\n",
       "   Profit           5.7624\n",
       "   profitRatio        0.49\n",
       "4  index              2468\n",
       "   Region             West\n",
       "   Customer ID    TP-21415\n",
       "   Sales             12.96\n",
       "   Profit           6.3504\n",
       "   profitRatio        0.49\n",
       "dtype: object"
      ]
     },
     "execution_count": 249,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sampStackData.stack()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 250,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>index</th>\n",
       "      <th>Region</th>\n",
       "      <th>Customer ID</th>\n",
       "      <th>Sales</th>\n",
       "      <th>Profit</th>\n",
       "      <th>profitRatio</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2206</td>\n",
       "      <td>West</td>\n",
       "      <td>LB-16735</td>\n",
       "      <td>21.48</td>\n",
       "      <td>10.74</td>\n",
       "      <td>0.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2367</td>\n",
       "      <td>West</td>\n",
       "      <td>RD-19810</td>\n",
       "      <td>146.82</td>\n",
       "      <td>73.41</td>\n",
       "      <td>0.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2422</td>\n",
       "      <td>West</td>\n",
       "      <td>SM-20320</td>\n",
       "      <td>9.96</td>\n",
       "      <td>4.8804</td>\n",
       "      <td>0.49</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1939</td>\n",
       "      <td>West</td>\n",
       "      <td>CG-12040</td>\n",
       "      <td>11.76</td>\n",
       "      <td>5.7624</td>\n",
       "      <td>0.49</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2468</td>\n",
       "      <td>West</td>\n",
       "      <td>TP-21415</td>\n",
       "      <td>12.96</td>\n",
       "      <td>6.3504</td>\n",
       "      <td>0.49</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  index Region Customer ID   Sales  Profit profitRatio\n",
       "0  2206   West    LB-16735   21.48   10.74         0.5\n",
       "1  2367   West    RD-19810  146.82   73.41         0.5\n",
       "2  2422   West    SM-20320    9.96  4.8804        0.49\n",
       "3  1939   West    CG-12040   11.76  5.7624        0.49\n",
       "4  2468   West    TP-21415   12.96  6.3504        0.49"
      ]
     },
     "execution_count": 250,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sampStackData.stack().unstack()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Conversion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Cast a pandas object to a specified dtype dtype."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 251,
   "metadata": {},
   "outputs": [],
   "source": [
    "xData=pd.DataFrame(data={'col1':[1,2,3,4]})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 252,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "col1    int64\n",
       "dtype: object"
      ]
     },
     "execution_count": 252,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "xData.dtypes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 253,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>col1</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   col1\n",
       "0     1\n",
       "1     2\n",
       "2     3\n",
       "3     4"
      ]
     },
     "execution_count": 253,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "xData.astype('int32')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 254,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "col1    int32\n",
       "dtype: object"
      ]
     },
     "execution_count": 254,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "xData.astype('int32').dtypes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 255,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>col1</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  col1\n",
       "0    1\n",
       "1    2\n",
       "2    3\n",
       "3    4"
      ]
     },
     "execution_count": 255,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "xData.astype('category')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 256,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "col1    category\n",
       "dtype: object"
      ]
     },
     "execution_count": 256,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "xData.astype('category').dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- to_datetime\n",
    "- to_timedelta\n",
    "- to_numeric\n",
    "- numpy.ndarray.astype"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Operation with Strings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 257,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Row ID</th>\n",
       "      <th>Order ID</th>\n",
       "      <th>Order Date</th>\n",
       "      <th>Ship Date</th>\n",
       "      <th>Ship Mode</th>\n",
       "      <th>Customer ID</th>\n",
       "      <th>Customer Name</th>\n",
       "      <th>Segment</th>\n",
       "      <th>Country</th>\n",
       "      <th>City</th>\n",
       "      <th>...</th>\n",
       "      <th>Postal Code</th>\n",
       "      <th>Region</th>\n",
       "      <th>Product ID</th>\n",
       "      <th>Category</th>\n",
       "      <th>Sub-Category</th>\n",
       "      <th>Product Name</th>\n",
       "      <th>Sales</th>\n",
       "      <th>Quantity</th>\n",
       "      <th>Discount</th>\n",
       "      <th>Profit</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>CA-2016-152156</td>\n",
       "      <td>2016-11-08</td>\n",
       "      <td>2016-11-11</td>\n",
       "      <td>Second Class</td>\n",
       "      <td>CG-12520</td>\n",
       "      <td>Claire Gute</td>\n",
       "      <td>Consumer</td>\n",
       "      <td>United States</td>\n",
       "      <td>Henderson</td>\n",
       "      <td>...</td>\n",
       "      <td>42420</td>\n",
       "      <td>South</td>\n",
       "      <td>FUR-BO-10001798</td>\n",
       "      <td>Furniture</td>\n",
       "      <td>Bookcases</td>\n",
       "      <td>Bush Somerset Collection Bookcase</td>\n",
       "      <td>261.96</td>\n",
       "      <td>2</td>\n",
       "      <td>0.0</td>\n",
       "      <td>41.9136</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1 rows × 21 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   Row ID        Order ID Order Date  Ship Date     Ship Mode Customer ID  \\\n",
       "0       1  CA-2016-152156 2016-11-08 2016-11-11  Second Class    CG-12520   \n",
       "\n",
       "  Customer Name   Segment        Country       City  ... Postal Code  Region  \\\n",
       "0   Claire Gute  Consumer  United States  Henderson  ...       42420   South   \n",
       "\n",
       "        Product ID   Category Sub-Category                       Product Name  \\\n",
       "0  FUR-BO-10001798  Furniture    Bookcases  Bush Somerset Collection Bookcase   \n",
       "\n",
       "    Sales  Quantity  Discount   Profit  \n",
       "0  261.96         2       0.0  41.9136  \n",
       "\n",
       "[1 rows x 21 columns]"
      ]
     },
     "execution_count": 257,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "salesData.head(1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 259,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0         Second Class\n",
       "1         Second Class\n",
       "2         Second Class\n",
       "3       Standard Class\n",
       "4       Standard Class\n",
       "             ...      \n",
       "9989      Second Class\n",
       "9990    Standard Class\n",
       "9991    Standard Class\n",
       "9992    Standard Class\n",
       "9993      Second Class\n",
       "Name: Ship Mode, Length: 9994, dtype: object"
      ]
     },
     "execution_count": 259,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "salesData['Ship Mode']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 258,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0      [Second, Class]\n",
       "1      [Second, Class]\n",
       "2      [Second, Class]\n",
       "3    [Standard, Class]\n",
       "4    [Standard, Class]\n",
       "Name: Ship Mode, dtype: object"
      ]
     },
     "execution_count": 258,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "salesData['Ship Mode'].str.split(' ').head(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 260,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    True\n",
       "1    True\n",
       "2    True\n",
       "3    True\n",
       "4    True\n",
       "Name: Ship Mode, dtype: bool"
      ]
     },
     "execution_count": 260,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "salesData['Ship Mode'].str.contains(' ').head(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 261,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    True\n",
       "1    True\n",
       "2    True\n",
       "3    True\n",
       "4    True\n",
       "Name: Ship Mode, dtype: bool"
      ]
     },
     "execution_count": 261,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "salesData['Ship Mode'].str.contains('Class').head(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 262,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0      Second$Class\n",
       "1      Second$Class\n",
       "2      Second$Class\n",
       "3    Standard$Class\n",
       "4    Standard$Class\n",
       "Name: Ship Mode, dtype: object"
      ]
     },
     "execution_count": 262,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "salesData['Ship Mode'].str.replace(' ','$').head(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 263,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     True\n",
       "1     True\n",
       "2     True\n",
       "3    False\n",
       "4    False\n",
       "Name: Ship Mode, dtype: bool"
      ]
     },
     "execution_count": 263,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "salesData['Ship Mode'].str.startswith('Sec').head(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}