KNN_FROM_SCRATCH.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Understanding KNN"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Outlines\n",
    "* What is KNN ?\n",
    "* What is Euclidean Distance?\n",
    "* Mathematical equation for Euclidean Distance\n",
    "* Building our KNN from scratch\n",
    "* Applying our model to real world dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# What is KNN?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "K-Nearest Neighbors (KNN) is one of the simplest algorithms used in Machine Learning for regression and classification problem. KNN algorithms use data and classify new data points based on similarity measures (e.g. distance function).\n",
    "\n",
    "Classification is done by a majority vote to its neighbors. The data is assigned to the class which has the nearest neighbors. As you increase the number of nearest neighbors, the value of k, accuracy might increase"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  KNN\n",
    "\n",
    "![](knn.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# What Is Euclidean Distance?\n",
    "\n",
    "Euclidean distance is the distance between two points. To find the two points on a plane, the length of a segment connecting the two points is measured\n",
    "\n",
    "The Euclidean distance formula, as its name suggests, gives the distance between two points (or) the straight line distance. Let us assume that (x1,y1) and (x2,y2) are two points in a two-dimensional plane."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Mathematical Expression for Euclidean distance\n",
    "\n",
    "\n",
    "d = √[ (x2 – x1)2 + (y2 – y1)2]\n",
    "\n",
    "\n",
    "![](euclidean.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "KNN calculates the Euclidean distance between two points "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Building KNN "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 158,
   "metadata": {},
   "outputs": [],
   "source": [
    "from math import sqrt\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "from collections import Counter\n",
    "from matplotlib import style\n",
    "style.use('fivethirtyeight')\n",
    "import seaborn as sns\n",
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 159,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5.0"
      ]
     },
     "execution_count": 159,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#let us assume these two points;\n",
    "p1 = (1,4)\n",
    "p2 = (4,8)\n",
    "\n",
    "Euclidean_distance = sqrt(((p2[0]- p1[0])**2)+ ((p2[1]-p1[1])**2))\n",
    "Euclidean_distance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 160,
   "metadata": {},
   "outputs": [],
   "source": [
    "#using an imaginary data\n",
    "dataset = {'k':[[3,2],[2,1],[1,3]], 'r':[[5,6],[6,7],[7,8]]}\n",
    "new = [4,6]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 161,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZIAAAEJCAYAAAC+I6F6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAZlUlEQVR4nO3df2xV9f3H8Vdva7lWMMVyU+MA+QK3Yu9A4RJli6KLo0DoegnL8FLrjD8HaBbicIhTUePUiDpjNwy/XJC23DFAc1t/kG5hK8ygpFuCu8uy2zQlGAKW68q+pdyx3nu/f6h8V8F7L3zOubf38Hwk/NFzP7fn9bbI655z7ukt6uvrSwkAgAvkyncAAEBho0gAAEYoEgCAEYoEAGCEIgEAGKFIAABGKBIAgBGKBABgZFgWSTQazXcE2zFj4XP6fBIzOoXdMw7LIgEAFA6KBABghCIBABjJWCSJRELPPvuspk2bpsrKSk2bNk3PPvusBgcHc5EPAHAhEgmVvPWWyhYvVtXSpSpbvFglb78tJZOW76ok04JXX31VmzZt0uuvv67q6mpFIhEtW7ZMpaWl+ulPf2p5IACAmaLeXpUFgyqORFQUj+uSL7aXdHQo0diogVBIKY/Hsv1lLJKPPvpI8+bN0/z58yVJV199tebPn6/Ozk7LQgAALJJMqiwYVMk5/o0uisdV0tmpsmBQJ9vbJZc1VzcyfpdZs2Zp3759+sc//iFJ+vvf/669e/dqzpw5lgQAAFinJBxWcSSSdk1xJKKStjbr9plpwYoVK9Tf368bb7xRxcXFGhwc1MqVK3XfffdZFgIAYI3SlhYVxeNp1xTF4yptatJgXZ0l+8xYJLt27VIoFNKmTZs0ZcoUffzxx3r00Uc1fvx4/fCHP/za55neAMNNQs7g9BmdPp/EjIWmKhY7c00knVOxWNZze73etI9nLJInn3xSDz30kL7//e9Lknw+nw4fPqxf/OIXaYsk047TiUajRs8vBMxY+Jw+n8SMhejSioqs11k1d8ZrJAMDAyouLh6yrbi4WEkb3kIGADBzur5eKbc77ZqU263TDQ2W7TNjkcybN0+vvvqqdu/erUOHDqm1tVW/+tWvVFtba1kIAIA1BuvqlPD50q5J+HwatPDf8Iyntl588UX9/Oc/109+8hMdP35clZWVuuuuu7iHBACGI5dLA6HQkPtIvpRyu5Xw+TQQCln21l8piyIZNWqUXnjhBb3wwguW7RQAYJ+Ux6OT7e0qaW1VaXOzTsViurSiQqcbGj4/ErGwRKQsigQAUIBcLg0GAhoMBGx/QwG/tBEAYIQiAQAYoUgAAEYoEgCAEYoEAGCEIgEAGKFIAABGKBIAgBGKBABghCIBABihSAAARigSAIARigQAYIQiAQAYoUgAAEYoEgCAEYoEAGCEIgEAGOGjdgFcfBIJlYTDKt22TVVffp55fb0G6+os/zzzi0HGIpk6daoOHz581vaamhpt377dllAAYJei3l6VBYMqjkRUFI/rki+2l3R0KNHYqIFQSCmPJ68ZC03GItmzZ48SicSZr48ePapbb71VCxcutDUYAFgumVRZMKiSzs6zHiqKx1XS2amyYFAn29s5MjkPGf9LjRkzRpWVlWf+tLe3a9SoURQJgIJTEg6rOBJJu6Y4ElFJW1uOEjnDeVVuKpXS1q1bdfvtt6usrMyuTABgi9KWFhXF42nXFMXjKm1qylEiZzivi+179uzRoUOHdOedd2ZcG41GLziUFc8vBMxY+Jw+n+SsGatisTPXRNI5FYs5am7J7Ofo9XrTPn5eRbJlyxbNmDFD06ZNM95xOtFo1Oj5hYAZC5/T55OcN+OlFRVZr3PS3Hb/HLM+tdXb26t3331Xd911l21hAMBOp+vrlXK7065Jud063dCQo0TOkHWRNDc3a8SIEVq0aJGdeQDANoN1dUr4fGnXJHw+DdbW5iiRM2RVJKlUSm+++aYWLVqkUaNG2Z0JAOzhcmkgFNKg33/WkUnK7dag36+BUIi3/p6nrK6R7N27V93d3dq4caPdeQDAVimPRyfb21XS2qrS5mad+vLO9oaGz49EKJHzllWRzJ49W319fXZnAYDccLk0GAhoMBBw3BsK8oHqBQAYoUgAAEYoEgCAEYoEAGCEIgEAGKFIAABGKBIAgBGKBABghCIBABihSAAARigSAIARigQAYIQiAQAYoUgAAEYoEgCAEYoEAGCEIgEAGKFIAABGsvqoXQCfSyRTCh86pW1dA4r9a4Qqeo6rfnKZ6iZcKldRUb7jWSORUEk4rNJt21T15eeZ19drsK6OzzPHOWVVJEePHtVTTz2l9vZ29ff3a8KECXr55Zd100032Z0PGDZ6TyUU/F1Mkc/+o3hSkoqlf/1bHUf+rca/9iv03Qp5Li3Od0wjRb29KgsGVRyJqCge1yVfbC/p6FCisVEDoZBSHk9eM2L4yfjyoq+vT3PnzlUqldL27dv14Ycf6sUXX5SHv0y4iCRTKQV/F1Pn8S9L5P/Fk1Ln8f8o+LuYkqlUfgJaIZlUWTCoks5OFcXjQx4qisdV0tmpsmBQSia/5hvgYpXxiOS1117TlVdeqfXr15/ZNmHCBDszAcNOuOeUIp/9J+2ayGf/UduhU6qbUJajVNYqCYdVHImkXVMciaikre3z01zAFzIekbzzzjvy+/26++67NXnyZN10003asGGDUoX8ygs4Ty1dA2cdiXxVPCk1RQdyE8gGpS0tZx2JfFVRPK7SpqYcJUKhyHhE0tPTo82bN2v58uVasWKFPv74Y61atUqS9MADD3zt86LRqFEw0+cXAmYsHLF/jZCU+fpH7H8HCnbmqljszDWRdE7FYgU749dx2jznYjKj1+tN+3jGIkkmk5o+fbrWrFkjSbruuuvU3d2tTZs2pS2STDtOJxqNGj2/EDBjYanoOS7969+Z140qk9c7PgeJrHdpRUXW65zyc5Wc9ff069g9Y8ZTW5WVlbrmmmuGbKuqqtInn3xiWyhguKmfXCZ3hv9b3C6pwVuY10ck6XR9vVJud9o1KbdbpxsacpQIhSJjkcyaNUtdXV1DtnV1dWncuHG2hQKGm7oJl8p3RfoTP74rLlHt1ZfmKJH1BuvqlPD50q5J+HwarK3NUSIUioxFsnz5ch04cEAvvfSSuru79fbbb2vDhg267777cpEPGBZcRUUKfbdC/jGXnHVk4nZJ/jGXKPTdisK+KdHl0kAopEG//6wjk5TbrUG/XwOhEDcl4iwZr5HMmDFDzc3NeuaZZ7R27VqNHTtWjz32GEWCi47n0mK113rU2nNKzV0Div3vgCpGlanBW6baq51xZ3vK49HJ9naVtLaqtLlZp768s72h4fMjEUoE55DVne1z587V3Llz7c4CDHuuoiIF/qdMgf8p++ICZmFeWE/L5dJgIKDBQOCiuBANc7y8AAAYoUgAAEYoEgCAEYoEAGCEIgEAGKFIAABGKBIAgBGKBABghCIBABihSAAARigSAIARigQAYIQiAQAYoUgAAEYoEgCAEYoEAGCEIgEAGKFIAABGKBIAgJGMRfL888+rvLx8yJ+qqqpcZAMAFICSbBZ5vV61tbWd+bq4uNi2QACAwpJVkZSUlKiystLuLACAApTVNZKenh5de+21mjZtmu655x719PTYHAsAUCgyFsnMmTO1bt06/fa3v9Vrr72mY8eOqaamRp999lku8gEAhrmivr6+1Pk8ob+/X9dff71WrFihhx566GvXRaNR43AAgPzzer1pH8/qGsl/GzlypKZMmaLu7m6jHacTjUaNnl8ImLHwOX0+iRmdwu4Zz/s+kng8rmg0ysV3AICkLI5IHn/8cc2bN09jx47V8ePHtXbtWg0MDGjJkiW5yAcAGOYyFsmRI0d03333KRaLacyYMZo5c6ba29s1fvz4XOQDAAxzGYvkjTfeyEUOAECB4ndtAQCMUCQAACMUCQDACEUCADBCkQAAjFAkAAAjFAkAwAhFAgAwQpEAAIxQJAAAIxQJAMAIRQIAMEKRAACMUCQAACMUCQDACEUCADBCkQAAjFAkAAAjFAkAwMh5F8nLL7+s8vJyPfLII3bkAQAUmPMqkgMHDmjLli3y+Xx25QEAFJisi+TEiRO6//771djYqPLycjszAQAKSNZFsmLFCgUCAd1yyy125gEAFJiSbBZt2bJF3d3dWr9+vd15AAAFpqivry+VbkE0GtW8efP03nvvqaqqSpK0YMECVVdXa+3atWmfBwAofF6vN+3jGYukublZDz74oIqLi89sSyQSKioqksvl0pEjRzRixAhr0n4hGo1mDF7omLHwOX0+iRmdwu4ZM57aWrBggaZPnz5k24MPPqhJkybp4YcfVmlpqW3hAADDX8YiKS8vP+tdWmVlZRo9erSqq6ttCwYAKAzc2Q4AMJLVu7a+6p133rE6BwCgQHFEAgAwQpEAAIxQJAAAIxQJAMAIRQIAMEKRAACMUCQAACMUCQDACEUCADBCkQAAjFAkAAAjFAkAwAhFAgAwQpEAAIxQJAAAIxQJAMAIRQIAMEKRAACMUCQAACMX9JntdkgkEgqHw9q2bZtisZgqKipUX1+vuro6uVz0HQAMVxmLZOPGjfr1r3+tw4cPS5KmTJmilStXau7cuZaF6O3tVTAYVCQSUTweP7O9o6NDjY2NCoVC8ng8lu0PAGCdjC/1r7rqKj399NP64x//qD179mj27Nm644479Ne//tWSAMlkUsFgUJ2dnUNKRJLi8bg6OzsVDAaVTCYt2R8AwFoZi2TBggWaM2eOJk6cqMmTJ+uJJ57QyJEjdeDAAUsChMNhRSKRtGsikYja2tos2R8AwFrndfEhkUho586dOnnypG644QZLArS0tJx1JPJV8XhcTU1NluwPAGCtrC62RyIR1dTUKB6P67LLLlNTU5N8Pl/a50Sj0awCxGKxrNdl+z0LhdPmORenz+j0+SRmdAqTGb1eb9rHsyoSr9ervXv36sSJEwqHw1q2bJna2tpUXV19wTv+UkVFRdbrsv2ehSAajTpqnnNx+oxOn09iRqewe8asTm2VlpZq4sSJmj59utasWaOpU6dq3bp1lgSor6+X2+1Ou8btdquhocGS/QEArHVBN2gkk0mdPn3akgB1dXUZT5P5fD7V1tZasj8AgLUyFslTTz2lDz74QIcOHVIkEtHTTz+tffv26Qc/+IE1AVwuhUIh+f3+s45M3G63/H6/QqEQNyUCwDCV8RrJsWPH9MADD+jTTz/V5ZdfLp/Ppx07dui2226zLITH41F7e7taW1vV3Nx85s72hoYG1dbWUiIAMIxlLJLXX389FznkcrkUCAQUCAQuiotfAOAUvNQHABihSAAARigSAIARigQAYIQiAQAYoUgAAEYoEgCAEYoEAGCEIgEAGKFIAABGKBIAgBGKBABghCIBABihSAAARigSAIARigQAYIQiAQAYoUgAAEYyftQucD4SiYTC4bC2bdumWCymiooK1dfXq66uTi4Xr1sAJ8pYJK+88opaW1vV1dWl0tJSzZw5U2vWrFF1dXUu8qGA9Pb2KhgMKhKJKB6Pn9ne0dGhxsZGhUIheTyePCYEYIeMLxH37dune++9V7t371Y4HFZJSYkWLlyof/7zn7nIhwKRTCYVDAbV2dk5pEQkKR6Pq7OzU8FgUMlkMk8JAdgl4xHJrl27hny9fv16jR8/Xvv379f8+fNtC4bCEg6HFYlE0q6JRCJqa2tTXV1djlIByIXzPmnd39+vZDKp8vJyO/KgQLW0tJx1JPJV8XhcTU1NOUoEIFfO+2L7o48+qqlTp+qGG25Iuy4ajV5wKCueXwicNGMsFst6nZPmdtIsX4cZncFkRq/Xm/bx8yqSxx57TPv379f777+v4uJiox2nE41GjZ5fCJw2Y0VFRdbrnDK3036G58KMzmD3jFmf2lq9erV27typcDisCRMm2BYIham+vl5utzvtGrfbrYaGhhwlApArWRXJqlWrtGPHDoXDYVVVVdmdCQWorq5OPp8v7Rqfz6fa2tocJQKQKxmLZOXKlWppadGmTZtUXl6uY8eO6dixY+rv789FPhQIl8ulUCgkv99/1pGJ2+2W3+9XKBTipkTAgTJeI9m0aZMkKRAIDNm+atUqrV692p5UKEgej0ft7e1qbW1Vc3PzmTvbGxoaVFtbS4kADpWxSPr6+nKRAw7hcrkUCAQUCAQuiouYAPiljQAAQxQJAMAIRQIAMEKRAACMUCQAACMUCQDACEUCADBCkQAAjFAkAAAjFAkAwAhFAgAwQpEAAIxQJAAAIxQJAMAIRQIAMEKRAACMUCQAACMUCQDACEUCADCSVZH86U9/UjAY1LXXXqvy8nI1NzfbncuREomE3nrrLS1evFhLly7V4sWL9fbbbyuZTOY7GgBcsJJsFp08eVLV1dVasmSJli5dancmR+rt7VUwGFQkElE8Hj+zvaOjQ42NjQqFQvJ4PHlMCAAXJqsjkpqaGj355JMKBAJyuTgbdr6SyaSCwaA6OzuHlIgkxeNxdXZ2KhgMcmQCoCDRCjkQDocViUTSrolEImpra8tRIgCwDkWSAy0tLWcdiXxVPB5XU1NTjhIBgHWyukZyIaLRaF6fP5zEYrGs1zlpbslZP8dzcfp8EjM6hcmMXq837eO2FUmmHacTjUaNnj/cVFRUZL3OSXM77ef4VU6fT2JGp7B7Rk5t5UB9fb3cbnfaNW63Ww0NDTlKBADWyapI+vv7dfDgQR08eFDJZFKffPKJDh48qMOHD9udzxHq6urk8/nSrvH5fKqtrc1RIgCwTlZF8pe//EWzZ8/W7NmzderUKT3//POaPXu2nnvuObvzOYLL5VIoFJLf7z/ryMTtdsvv9ysUCvHWagAFKatrJDfffLP6+vrszuJoHo9H7e3tam1tVXNzs2KxmCoqKtTQ0KDa2lpKBEDBsu1iO87mcrkUCAQUCAQuigt8AC4OvAwGABihSAAARor6+vpS+Q4BAChcHJEAAIxQJAAAIxQJAMAIRQIAMEKRAACMDJsicfrnwr/yyiv6zne+o3HjxmnSpEm6/fbb9be//S3fsSy1ceNGffvb39a4ceM0btw4zZkzR7t37853LFu9/PLLKi8v1yOPPJLvKJZ5/vnnVV5ePuRPVVVVvmNZ7ujRo1q6dKkmTZqkyspK3Xjjjdq3b1++Y1lm6tSpZ/0cy8vLtXjxYsv3NWzubHf658Lv27dP9957r2bMmKFUKqXnnntOCxcu1IcffqjRo0fnO54lrrrqKj399NOaNGmSksmktm3bpjvuuEN/+MMf9M1vfjPf8Sx34MABbdmyJeMv5CxEXq93yCd2FhcX5zGN9fr6+jR37lzNmjVL27dvV0VFhQ4dOiSPx5PvaJbZs2ePEonEma+PHj2qW2+9VQsXLrR8X8OmSGpqalRTUyNJWr58eZ7TWG/Xrl1Dvl6/fr3Gjx+v/fv3a/78+XlKZa0FCxYM+fqJJ57Q5s2bdeDAAccVyYkTJ3T//fersbFRL774Yr7jWK6kpESVlZX5jmGb1157TVdeeaXWr19/ZtuECRPyF8gGY8aMGfL11q1bNWrUKFuKZNic2rrY9Pf3K5lMqry8PN9RbJFIJLRz506dPHlSN9xwQ77jWG7FihUKBAK65ZZb8h3FFj09Pbr22ms1bdo03XPPPerp6cl3JEu988478vv9uvvuuzV58mTddNNN2rBhg1IpZ96fnUqltHXrVt1+++0qKyuz/PsPmyOSi82jjz6qqVOnOu4f2UgkopqaGsXjcV122WVqampy3KmfLVu2qLu7e8irWSeZOXOm1q1bJ6/Xq+PHj2vt2rWqqanR/v37dcUVV+Q7niV6enq0efNmLV++XCtWrNDHH3+sVatWSZIeeOCBPKez3p49e3To0CHdeeedtnx/iiQPHnvsMe3fv1/vv/++4849e71e7d27VydOnFA4HNayZcvU1tam6urqfEezRDQa1TPPPKP33ntPpaWl+Y5jizlz5gz5eubMmbr++uvV0tKihx56KE+prJVMJjV9+nStWbNGknTdddepu7tbmzZtcmSRbNmyRTNmzNC0adNs+f4USY6tXr1au3btUmtrq+POyUpSaWmpJk6cKEmaPn26/vznP2vdunX65S9/medk1vjoo48Ui8X0rW9968y2RCKhDz74QG+88YaOHDmiESNG5DGh9UaOHKkpU6aou7s731EsU1lZqWuuuWbItqqqKn3yySd5SmSf3t5evfvuu3rppZds2wdFkkOrVq3Srl271NbW5si3U55LMpnU6dOn8x3DMgsWLND06dOHbHvwwQc1adIkPfzww448SonH44pGo7r55pvzHcUys2bNUldX15BtXV1dGjduXJ4S2ae5uVkjRozQokWLbNvHsCmS/v7+M694/vtz4UePHu2IH+7KlSv1m9/8Rk1NTSovL9exY8ckSZdddplGjhyZ53TWeOqpp1RTU6NvfOMb6u/v144dO7Rv3z5t374939Es8+V78f9bWVmZRo8e7ZjTd48//rjmzZunsWPHnrlGMjAwoCVLluQ7mmWWL1+umpoavfTSS1q0aJEOHjyoDRs26Iknnsh3NEulUim9+eabWrRokUaNGmXbfobNr5Hfu3evvve97521fcmSJXr99dfzkMhaX/furFWrVmn16tU5TmOPZcuWae/evfr00091+eWXy+fz6cc//rFuu+22fEez1YIFC1RdXa21a9fmO4ol7rnnHn3wwQeKxWIaM2aMZs6cqZ/97GeaMmVKvqNZavfu3XrmmWfU1dWlsWPH6v7779ePfvQjFRUV5TuaZTo6OlRXV6ff//738vv9tu1n2BQJAKAwcR8JAMAIRQIAMEKRAACMUCQAACMUCQDACEUCADBCkQAAjFAkAAAjFAkAwMj/AbLT6xre5W4yAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "for d in dataset:\n",
    "    for dd in dataset[d]:\n",
    "        plt.scatter(dd[0],dd[1], s= 100, color = d)\n",
    "plt.scatter(new[0],new[1], s= 100)        \n",
    "plt.show()        "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Just looking at the above graph, we could easily say the blue point belongs to the red group as it close to them i.e their neighbor, however, let us apply our KNN algorithm to decide the group it belongs "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 162,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "r\n"
     ]
    }
   ],
   "source": [
    "def k_nearest_neighbor(data, predict, k=3):\n",
    "    distances = []\n",
    "    for d in data:\n",
    "        for features in data[d]:\n",
    "            #euclidean distance = sqrt((p1[0]-p2[0])**2)+((p1[1]-p2[1])**2)\n",
    "            euclidean_distance = np.linalg.norm(np.array(features)- np.array(predict))\n",
    "            distances.append([euclidean_distance, d])\n",
    "     \n",
    "    votes = [d[1] for d in sorted(distances)[:k]]\n",
    "    vote_result = Counter(votes).most_common(1)[0][0]\n",
    "    return vote_result\n",
    "\n",
    "result = k_nearest_neighbor(dataset, new, k=3)\n",
    "print(result)\n",
    "        \n",
    "        "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first formula for euclidean distance(which is commented) is the basic formula for calculating euclidean distance, and only allows limited features(2), while the second allows for more than 2 features, both formula will be used for predictions for two features and three features respectively"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 163,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZIAAAEJCAYAAAC+I6F6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAYoElEQVR4nO3df2zU9R3H8VevXbkVMGWlqXHACHAVegOFEmQLoouhQOh6hGVw1DojAuNXlsbhEKeCxqkRcYZuEH65IG25MUBzLSppFjZgBCXdEtgty440EAgBy20lKXAhvbv9MSWr6N3B5/u96337fCT80e99rt/X25q+7vv93reX19XVlRAAAHfJle0AAIDcRpEAAIxQJAAAIxQJAMAIRQIAMEKRAACMUCQAACMUCQDASJ8sknA4nO0ItmPG3Of0+SRmdAq7Z+yTRQIAyB0UCQDACEUCADCSskhisZheffVVTZgwQWVlZZowYYJeffVV9fT0ZCIfAOBuxGIqeP99Fc2fr/Jly1Q0f74KPvhAisct31VBqgXvvPOOduzYoS1btqiiokKhUEjLly9XYWGhfvGLX1geCABgJq+zU0V+v/JDIeVFo/rG59sLjhxRrKFB1wMBJUpLLdtfyiL59NNPNWvWLM2ePVuS9J3vfEezZ89We3u7ZSEAABaJx1Xk96vgK35H50WjKmhvV5Hfr2ttbZLLmqsbKb/L1KlTdezYMf3rX/+SJP3zn//U0aNHNWPGDEsCAACsUxAMKj8USromPxRSQWurdftMtaC+vl7d3d166KGHlJ+fr56eHq1evVqLFy+2LAQAwBqFzc3Ki0aTrsmLRlXY2KiemhpL9pmySA4cOKBAIKAdO3Zo7NixOn36tJ577jmNGDFCP/nJT772eaY3wHCTkDM4fUanzycxY64pj0RuXRNJ5kYkkvbcHo8n6eMpi+Sll17SqlWr9KMf/UiS5PV6df78ef36179OWiSpdpxMOBw2en4uYMbc5/T5JGbMRd8sKUl7nVVzp7xGcv36deXn5/falp+fr7gNbyEDAJi5WVurhNuddE3C7dbNujrL9pmySGbNmqV33nlHhw4d0rlz59TS0qLf/va3qq6utiwEAMAaPTU1inm9SdfEvF71WPg7POWprTfffFO/+tWv9POf/1xXrlxRWVmZnnzySe4hAYC+yOXS9UCg130kX0i43Yp5vboeCFj21l8pjSIZPHiw3njjDb3xxhuW7RQAYJ9EaamutbWpoKVFhU1NuhGJ6JslJbpZV/e/IxELS0RKo0gAADnI5VKPz6cen8/2NxTwRxsBAEYoEgCAEYoEAGCEIgEAGKFIAABGKBIAgBGKBABghCIBABihSAAARigSAIARigQAYIQiAQAYoUgAAEYoEgCAEYoEAGCEIgEAGKFIAABGKBIAgBE+ahdA/xOLqSAYVOGePSr/4vPMa2vVU1Nj+eeZ9wcpi2T8+PE6f/78bdurqqq0d+9eW0IBgF3yOjtV5PcrPxRSXjSqb3y+veDIEcUaGnQ9EFCitDSrGXNNyiI5fPiwYrHYra8vXbqkRx99VHPnzrU1GABYLh5Xkd+vgvb22x7Ki0ZV0N6uIr9f19raODK5Ayn/Sw0dOlRlZWW3/rW1tWnw4MEUCYCcUxAMKj8USromPxRSQWtrhhI5wx1VbiKR0O7du7VgwQIVFRXZlQkAbFHY3Ky8aDTpmrxoVIWNjRlK5Ax3dLH98OHDOnfunJ544omUa8Ph8F2HsuL5uYAZc5/T55OcNWN5JHLrmkgyNyIRR80tmf0cPR5P0sfvqEh27dqlSZMmacKECcY7TiYcDhs9PxcwY+5z+nyS82b8ZklJ2uucNLfdP8e0T211dnbqww8/1JNPPmlbGACw083aWiXc7qRrEm63btbVZSiRM6RdJE1NTRowYIDmzZtnZx4AsE1PTY1iXm/SNTGvVz3V1RlK5AxpFUkikdB7772nefPmafDgwXZnAgB7uFy6Hgiop7LytiOThNutnspKXQ8EeOvvHUrrGsnRo0fV0dGh7du3250HAGyVKC3VtbY2FbS0qLCpSTe+uLO9ru5/RyKUyB1Lq0imT5+urq4uu7MAQGa4XOrx+dTj8znuDQXZQPUCAIxQJAAAIxQJAMAIRQIAMEKRAACMUCQAACMUCQDACEUCADBCkQAAjFAkAAAjFAkAwAhFAgAwQpEAAIxQJAAAIxQJAMAIRQIAMEKRAACMUCQAACNpfdQugM/FYioIBlW4Z4/Kv/is79pa9dTUOOezvvvDjLBUWkVy6dIlrV+/Xm1tberu7tbIkSO1ceNGTZs2ze58QJ+R19mpIr9f+aGQ8qJRfePz7QVHjijW0KDrgYASpaVZzWiqP8wI66V8edHV1aWZM2cqkUho7969+uSTT/Tmm2+qlP+Z0J/E4yry+1XQ3q68aLTXQ3nRqAra21Xk90vxeJYCWqA/zAhbpDwi2bRpk+69915t3br11raRI0famQnocwqCQeWHQknX5IdCKmht/d8poBzUH2aEPVIekRw8eFCVlZV66qmnNGbMGE2bNk3btm1TIpHIRD6gTyhsbr7tVfqX5UWjKmxszFAi6/WHGWGPlEckZ8+e1c6dO7VixQrV19fr9OnTWrNmjSRp6dKlX/u8cDhsFMz0+bmAGXNHeSRy63pBMjcikZyduT/M+HWcNs9XMZnR4/EkfTxlkcTjcU2cOFHr1q2TJD3wwAPq6OjQjh07khZJqh0nEw6HjZ6fC5gxt3yzpCTtdbk6c3+Y8as46f/Tr2P3jClPbZWVlen+++/vta28vFwXLlywLRTQ19ysrVXC7U66JuF262ZdXYYSWa8/zAh7pCySqVOn6syZM722nTlzRsOHD7ctFNDX9NTUKOb1Jl0T83rVU12doUTW6w8zwh4pi2TFihU6efKk3nrrLXV0dOiDDz7Qtm3btHjx4kzkA/oGl0vXAwH1VFbe9qo94Xarp7JS1wOB3L5hrz/MCFukvEYyadIkNTU16ZVXXtGGDRs0bNgwPf/88xQJ+p1EaamutbWpoKVFhU1NuvHFXd91df97le6AX7D9YUZYL60722fOnKmZM2fanQXo+1wu9fh86vH5nHuRtj/MCEvx8gIAYIQiAQAYoUgAAEYoEgCAEYoEAGCEIgEAGKFIAABGKBIAgBGKBABghCIBABihSAAARigSAIARigQAYIQiAQAYoUgAAEYoEgCAEYoEAGCEIgEAGKFIAABGUhbJ66+/ruLi4l7/ysvLM5ENAJADCtJZ5PF41Nraeuvr/Px82wIBAHJLWkVSUFCgsrIyu7MAAHJQWtdIzp49q3HjxmnChAlatGiRzp49a3MsAECuSFkkkydP1ubNm/WHP/xBmzZt0uXLl1VVVaV///vfmcgHAOjj8rq6uhJ38oTu7m49+OCDqq+v16pVq752XTgcNg4HAMg+j8eT9PG0rpH8v0GDBmns2LHq6Ogw2nEy4XDY6Pm5gBlzn9Pnk5jRKeye8Y7vI4lGowqHw1x8BwBISuOI5IUXXtCsWbM0bNgwXblyRRs2bND169e1cOHCTOQDAPRxKYvk4sWLWrx4sSKRiIYOHarJkyerra1NI0aMyEQ+AEAfl7JI3n333UzkAADkKP7WFgDACEUCADBCkQAAjFAkAAAjFAkAwAhFAgAwQpEAAIxQJAAAIxQJAMAIRQIAMEKRAACMUCQAACMUCQDACEUCADBCkQAAjFAkAAAjFAkAwAhFAgAwQpEAAIzccZFs3LhRxcXFevbZZ+3IAwDIMXdUJCdPntSuXbvk9XrtygMAyDFpF8nVq1e1ZMkSNTQ0qLi42M5MAIAcknaR1NfXy+fz6ZFHHrEzDwAgxxSks2jXrl3q6OjQ1q1b7c4DAMgxeV1dXYlkC8LhsGbNmqWPPvpI5eXlkqQ5c+aooqJCGzZsSPo8AEDu83g8SR9PWSRNTU1auXKl8vPzb22LxWLKy8uTy+XSxYsXNWDAAGvSfi4cDqcMnuuYMfc5fT6JGZ3C7hlTntqaM2eOJk6c2GvbypUrNXr0aD3zzDMqLCy0LRwAoO9LWSTFxcW3vUurqKhIQ4YMUUVFhW3BAAC5gTvbAQBG0nrX1pcdPHjQ6hwAgBzFEQkAwAhFAgAwQpEAAIxQJAAAIxQJAMAIRQIAMEKRAACMUCQAACMUCQDACEUCADBCkQAAjFAkAAAjFAkAwAhFAgAwQpEAAIxQJAAAIxQJAMAIRQIAMEKRAACM3NVnttshFospGAxqz549ikQiKikpUW1trWpqauRy0XcA0FelLJLt27frd7/7nc6fPy9JGjt2rFavXq2ZM2daFqKzs1N+v1+hUEjRaPTW9iNHjqihoUGBQEClpaWW7Q8AYJ2UL/Xvu+8+vfzyy/rzn/+sw4cPa/r06Xr88cf197//3ZIA8Xhcfr9f7e3tvUpEkqLRqNrb2+X3+xWPxy3ZHwDAWimLZM6cOZoxY4ZGjRqlMWPG6MUXX9SgQYN08uRJSwIEg0GFQqGka0KhkFpbWy3ZHwDAWnd08SEWi2n//v26du2apkyZYkmA5ubm245EviwajaqxsdGS/QEArJXWxfZQKKSqqipFo1ENHDhQjY2N8nq9SZ8TDofTChCJRNJel+73zBVOm+erOH1Gp88nMaNTmMzo8XiSPp5WkXg8Hh09elRXr15VMBjU8uXL1draqoqKirve8RdKSkrSXpfu98wF4XDYUfN8FafP6PT5JGZ0CrtnTOvUVmFhoUaNGqWJEydq3bp1Gj9+vDZv3mxJgNraWrnd7qRr3G636urqLNkfAMBad3WDRjwe182bNy0JUFNTk/I0mdfrVXV1tSX7AwBYK2WRrF+/XsePH9e5c+cUCoX08ssv69ixY/rxj39sTQCXS4FAQJWVlbcdmbjdblVWVioQCHBTIgD0USmvkVy+fFlLly7VZ599pnvuuUder1f79u3TY489ZlmI0tJStbW1qaWlRU1NTbfubK+rq1N1dTUlAgB9WMoi2bJlSyZyyOVyyefzyefz9YuLXwDgFLzUBwAYoUgAAEYoEgCAEYoEAGCEIgEAGKFIAABGKBIAgBGKBABghCIBABihSAAARigSAIARigQAYIQiAQAYoUgAAEYoEgCAEYoEAGCEIgEAGKFIAABGUn7ULnAnYrGYgsGg9uzZo0gkopKSEtXW1qqmpkYuF69bACdKWSRvv/22WlpadObMGRUWFmry5Mlat26dKioqMpEPOaSzs1N+v1+hUEjRaPTW9iNHjqihoUGBQEClpaVZTAjADilfIh47dkxPP/20Dh06pGAwqIKCAs2dO1f/+c9/MpEPOSIej8vv96u9vb1XiUhSNBpVe3u7/H6/4vF4lhICsEvKI5IDBw70+nrr1q0aMWKETpw4odmzZ9sWDLklGAwqFAolXRMKhdTa2qqampoMpQKQCXd80rq7u1vxeFzFxcV25EGOam5uvu1I5Mui0agaGxszlAhAptzxxfbnnntO48eP15QpU5KuC4fDdx3KiufnAifNGIlE0l7npLmdNMvXYUZnMJnR4/EkffyOiuT555/XiRMn9PHHHys/P99ox8mEw2Gj5+cCp81YUlKS9jqnzO20n+FXYUZnsHvGtE9trV27Vvv371cwGNTIkSNtC4TcVFtbK7fbnXSN2+1WXV1dhhIByJS0imTNmjXat2+fgsGgysvL7c6EHFRTUyOv15t0jdfrVXV1dYYSAciUlEWyevVqNTc3a8eOHSouLtbly5d1+fJldXd3ZyIfcoTL5VIgEFBlZeVtRyZut1uVlZUKBALclAg4UMprJDt27JAk+Xy+XtvXrFmjtWvX2pMKOam0tFRtbW1qaWlRU1PTrTvb6+rqVF1dTYkADpWySLq6ujKRAw7hcrnk8/nk8/n6xUVMAPzRRgCAIYoEAGCEIgEAGKFIAABGKBIAgBGKBABghCIBABihSAAARigSAIARigQAYIQiAQAYoUgAAEYoEgCAEYoEAGCEIgEAGKFIAABGKBIAgBGKBABghCIBABhJq0j+8pe/yO/3a9y4cSouLlZTU5PduRwpFovp/fff1/z587Vs2TLNnz9fH3zwgeLxeLajAcBdK0hn0bVr11RRUaGFCxdq2bJldmdypM7OTvn9foVCIUWj0Vvbjxw5ooaGBgUCAZWWlmYxIQDcnbSOSKqqqvTSSy/J5/PJ5eJs2J2Kx+Py+/1qb2/vVSKSFI1G1d7eLr/fz5EJgJxEK2RAMBhUKBRKuiYUCqm1tTVDiQDAOhRJBjQ3N992JPJl0WhUjY2NGUoEANZJ6xrJ3QiHw1l9fl8SiUTSXuekuSVn/Ry/itPnk5jRKUxm9Hg8SR+3rUhS7TiZcDhs9Py+pqSkJO11TprbaT/HL3P6fBIzOoXdM3JqKwNqa2vldruTrnG73aqrq8tQIgCwTlpF0t3drVOnTunUqVOKx+O6cOGCTp06pfPnz9udzxFqamrk9XqTrvF6vaqurs5QIgCwTlpF8re//U3Tp0/X9OnTdePGDb3++uuaPn26XnvtNbvzOYLL5VIgEFBlZeVtRyZut1uVlZUKBAK8tRpATkrrGsnDDz+srq4uu7M4Wmlpqdra2tTS0qKmpiZFIhGVlJSorq5O1dXVlAiAnGXbxXbczuVyyefzyefz9YsLfAD6B14GAwCMUCQAACN5XV1diWyHAADkLo5IAABGKBIAgBGKBABghCIBABihSAAARvpMkTj9c+Hffvtt/eAHP9Dw4cM1evRoLViwQP/4xz+yHctS27dv1/e//30NHz5cw4cP14wZM3To0KFsx7LVxo0bVVxcrGeffTbbUSzz+uuvq7i4uNe/8vLybMey3KVLl7Rs2TKNHj1aZWVleuihh3Ts2LFsx7LM+PHjb/s5FhcXa/78+Zbvq8/c2e70z4U/duyYnn76aU2aNEmJREKvvfaa5s6dq08++URDhgzJdjxL3HfffXr55Zc1evRoxeNx7dmzR48//rj+9Kc/6bvf/W6241nu5MmT2rVrV8o/yJmLPB5Pr0/szM/Pz2Ia63V1dWnmzJmaOnWq9u7dq5KSEp07d06lpaXZjmaZw4cPKxaL3fr60qVLevTRRzV37lzL99VniqSqqkpVVVWSpBUrVmQ5jfUOHDjQ6+utW7dqxIgROnHihGbPnp2lVNaaM2dOr69ffPFF7dy5UydPnnRckVy9elVLlixRQ0OD3nzzzWzHsVxBQYHKysqyHcM2mzZt0r333qutW7fe2jZy5MjsBbLB0KFDe329e/duDR482JYi6TOntvqb7u5uxeNxFRcXZzuKLWKxmPbv369r165pypQp2Y5jufr6evl8Pj3yyCPZjmKLs2fPaty4cZowYYIWLVqks2fPZjuSpQ4ePKjKyko99dRTGjNmjKZNm6Zt27YpkXDm/dmJREK7d+/WggULVFRUZPn37zNHJP3Nc889p/Hjxzvul2woFFJVVZWi0agGDhyoxsZGx5362bVrlzo6Onq9mnWSyZMna/PmzfJ4PLpy5Yo2bNigqqoqnThxQt/61reyHc8SZ8+e1c6dO7VixQrV19fr9OnTWrNmjSRp6dKlWU5nvcOHD+vcuXN64oknbPn+FEkWPP/88zpx4oQ+/vhjx5179ng8Onr0qK5evapgMKjly5ertbVVFRUV2Y5miXA4rFdeeUUfffSRCgsLsx3HFjNmzOj19eTJk/Xggw+qublZq1atylIqa8XjcU2cOFHr1q2TJD3wwAPq6OjQjh07HFkku3bt0qRJkzRhwgRbvj9FkmFr167VgQMH1NLS4rhzspJUWFioUaNGSZImTpyov/71r9q8ebN+85vfZDmZNT799FNFIhF973vfu7UtFovp+PHjevfdd3Xx4kUNGDAgiwmtN2jQII0dO1YdHR3ZjmKZsrIy3X///b22lZeX68KFC1lKZJ/Ozk59+OGHeuutt2zbB0WSQWvWrNGBAwfU2trqyLdTfpV4PK6bN29mO4Zl5syZo4kTJ/batnLlSo0ePVrPPPOMI49SotGowuGwHn744WxHsczUqVN15syZXtvOnDmj4cOHZymRfZqamjRgwADNmzfPtn30mSLp7u6+9Yrn/z8XfsiQIY744a5evVq///3v1djYqOLiYl2+fFmSNHDgQA0aNCjL6ayxfv16VVVV6dvf/ra6u7u1b98+HTt2THv37s12NMt88V78/1dUVKQhQ4Y45vTdCy+8oFmzZmnYsGG3rpFcv35dCxcuzHY0y6xYsUJVVVV66623NG/ePJ06dUrbtm3Tiy++mO1olkokEnrvvfc0b948DR482Lb99Jk/I3/06FH98Ic/vG37woULtWXLliwkstbXvTtrzZo1Wrt2bYbT2GP58uU6evSoPvvsM91zzz3yer362c9+psceeyzb0Ww1Z84cVVRUaMOGDdmOYolFixbp+PHjikQiGjp0qCZPnqxf/vKXGjt2bLajWerQoUN65ZVXdObMGQ0bNkxLlizRT3/6U+Xl5WU7mmWOHDmimpoa/fGPf1RlZaVt++kzRQIAyE3cRwIAMEKRAACMUCQAACMUCQDACEUCADBCkQAAjFAkAAAjFAkAwAhFAgAw8l/5tOh6d+CpVwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "for d in dataset:\n",
    "    for dd in dataset[d]:\n",
    "        plt.scatter(dd[0],dd[1], s= 100, color = d)\n",
    "plt.scatter(new[0],new[1], s= 100, color = result)        \n",
    "plt.show()        "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The above graph shows the result gotten from the KNN algorithm, the new point belongs the red group"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Applying KNN to real world predictions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Applying KNN to real world data, the dataset that will be used is Social_network_ads, it has a label column, [purchased] which tells whether or not a customer purchased a particular product, dataset can be gotten from https://www.kaggle.com/rakeshrau/social-network-ads"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 164,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import random"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 165,
   "metadata": {},
   "outputs": [],
   "source": [
    "ads = pd.read_csv('Social_network_ads.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 166,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>User ID</th>\n",
       "      <th>Gender</th>\n",
       "      <th>Age</th>\n",
       "      <th>EstimatedSalary</th>\n",
       "      <th>Purchased</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>15624510</td>\n",
       "      <td>Male</td>\n",
       "      <td>19</td>\n",
       "      <td>19000</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>15810944</td>\n",
       "      <td>Male</td>\n",
       "      <td>35</td>\n",
       "      <td>20000</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>15668575</td>\n",
       "      <td>Female</td>\n",
       "      <td>26</td>\n",
       "      <td>43000</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>15603246</td>\n",
       "      <td>Female</td>\n",
       "      <td>27</td>\n",
       "      <td>57000</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>15804002</td>\n",
       "      <td>Male</td>\n",
       "      <td>19</td>\n",
       "      <td>76000</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    User ID  Gender  Age  EstimatedSalary  Purchased\n",
       "0  15624510    Male   19            19000          0\n",
       "1  15810944    Male   35            20000          0\n",
       "2  15668575  Female   26            43000          0\n",
       "3  15603246  Female   27            57000          0\n",
       "4  15804002    Male   19            76000          0"
      ]
     },
     "execution_count": 166,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ads.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 167,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Female    204\n",
       "Male      196\n",
       "Name: Gender, dtype: int64"
      ]
     },
     "execution_count": 167,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ads['Gender'].value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 168,
   "metadata": {},
   "outputs": [],
   "source": [
    "ads['Gender'].replace('Male',1, inplace=True)\n",
    "ads['Gender'].replace('Female',2, inplace=True)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 169,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 170,
   "metadata": {},
   "outputs": [],
   "source": [
    "X= ads.iloc[:,2:4].values\n",
    "y= ads.iloc[:,-1].values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 171,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test ,y_train, y_test = train_test_split(X,y,test_size = 0.2, random_state = 1234)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 172,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import StandardScaler"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 173,
   "metadata": {},
   "outputs": [],
   "source": [
    "scaler = StandardScaler()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 174,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train = scaler.fit_transform(X_train)\n",
    "X_test = scaler.transform(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 175,
   "metadata": {},
   "outputs": [],
   "source": [
    "import operator\n",
    "from collections import Counter"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 176,
   "metadata": {},
   "outputs": [],
   "source": [
    "from math import sqrt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 177,
   "metadata": {},
   "outputs": [],
   "source": [
    "class KNearestNeighbors:\n",
    "    def __init__(self, k):\n",
    "        self.k = k\n",
    "        \n",
    "    def fit(self, X_train, y_train):\n",
    "        self.X_train = X_train\n",
    "        self.y_train = y_train\n",
    "        print('training done')\n",
    "        \n",
    "    def predict(self, X_test):\n",
    "        distance = {}\n",
    "        counter = 1\n",
    "        \n",
    "        for i in self.X_train:\n",
    "            distance[counter] = sqrt((X_test[0][0]-i[0])**2 + (X_test[0][1]- i[1])**2)\n",
    "            counter = counter + 1 \n",
    "        distance = sorted(distance.items(), key = operator.itemgetter(1))\n",
    "        self.classify(distance = distance[:self.k]) #when k is set to x, x number of distances are selected\n",
    "    \n",
    "    def classify (self,distance):\n",
    "        label = []\n",
    "        \n",
    "        for i in distance:\n",
    "            label.append(self.y_train[i[0]]) #this selects a corresponding label for the distances\n",
    "            \n",
    "        print(Counter(label).most_common()[0][0])  \n",
    "        \n",
    "        "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I created a class which has multiple functions, the predict function will take an array of two features ['age'] and ['Salary'] "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 178,
   "metadata": {},
   "outputs": [],
   "source": [
    "knn = KNearestNeighbors(k = 5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 179,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "training done\n"
     ]
    }
   ],
   "source": [
    "knn.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 180,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n"
     ]
    }
   ],
   "source": [
    "knn.predict(np.array([20, 5000]).reshape(1,2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Predicting on Multiple Features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this prediction we have an additional feature ['gender'] to ['age'] and ['salary'] "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Solar_network_ads dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 181,
   "metadata": {},
   "outputs": [],
   "source": [
    "ads = ads.astype(float).values.tolist()\n",
    "random.shuffle(ads)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 182,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_size = 0.2\n",
    "train_label = {0:[],1:[]}\n",
    "test_label = {0:[],1:[]}\n",
    "train_data = ads[:-int(test_size*len(ads))] #this takes all the data up to the 20% of len(dataset) i.e it takes 80% of the data\n",
    "test_data = ads[-int(test_size * len(ads)):]#this takes 20% of len(dataset)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 183,
   "metadata": {},
   "outputs": [],
   "source": [
    "for i in train_data:\n",
    "    train_label[i[-1]].append(i[:-1]) # here i am populating the training dictionary based on the labels 0 and 1  \n",
    "    \n",
    "for i in test_data:\n",
    "    test_label[i[-1]].append(i[:-1])\n",
    "        \n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 184,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy:  0.7625\n"
     ]
    }
   ],
   "source": [
    "correct = 0\n",
    "total = 0\n",
    "for group in test_label:\n",
    "    for data in test_label[group]:\n",
    "        vote = k_nearest_neighbor(train_label, data, k = 9)\n",
    "        if group == vote:\n",
    "            correct +=1\n",
    "        total +=1\n",
    "        \n",
    "print('accuracy: ', correct/total)        "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that the function k_nearest_neighbour has been defined earlier and was used for the initial classification of a point on the graph, the formula is;  euclidean_distance = np.linalg.norm(np.array(features)- np.array(predict)) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Breast_Cancer_winsconsin dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 185,
   "metadata": {},
   "outputs": [],
   "source": [
    "cancer = pd.read_csv('data.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 186,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>diagnosis</th>\n",
       "      <th>radius_mean</th>\n",
       "      <th>texture_mean</th>\n",
       "      <th>perimeter_mean</th>\n",
       "      <th>area_mean</th>\n",
       "      <th>smoothness_mean</th>\n",
       "      <th>compactness_mean</th>\n",
       "      <th>concavity_mean</th>\n",
       "      <th>concave points_mean</th>\n",
       "      <th>...</th>\n",
       "      <th>texture_worst</th>\n",
       "      <th>perimeter_worst</th>\n",
       "      <th>area_worst</th>\n",
       "      <th>smoothness_worst</th>\n",
       "      <th>compactness_worst</th>\n",
       "      <th>concavity_worst</th>\n",
       "      <th>concave points_worst</th>\n",
       "      <th>symmetry_worst</th>\n",
       "      <th>fractal_dimension_worst</th>\n",
       "      <th>Unnamed: 32</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>842302</td>\n",
       "      <td>M</td>\n",
       "      <td>17.99</td>\n",
       "      <td>10.38</td>\n",
       "      <td>122.80</td>\n",
       "      <td>1001.0</td>\n",
       "      <td>0.11840</td>\n",
       "      <td>0.27760</td>\n",
       "      <td>0.3001</td>\n",
       "      <td>0.14710</td>\n",
       "      <td>...</td>\n",
       "      <td>17.33</td>\n",
       "      <td>184.60</td>\n",
       "      <td>2019.0</td>\n",
       "      <td>0.1622</td>\n",
       "      <td>0.6656</td>\n",
       "      <td>0.7119</td>\n",
       "      <td>0.2654</td>\n",
       "      <td>0.4601</td>\n",
       "      <td>0.11890</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>842517</td>\n",
       "      <td>M</td>\n",
       "      <td>20.57</td>\n",
       "      <td>17.77</td>\n",
       "      <td>132.90</td>\n",
       "      <td>1326.0</td>\n",
       "      <td>0.08474</td>\n",
       "      <td>0.07864</td>\n",
       "      <td>0.0869</td>\n",
       "      <td>0.07017</td>\n",
       "      <td>...</td>\n",
       "      <td>23.41</td>\n",
       "      <td>158.80</td>\n",
       "      <td>1956.0</td>\n",
       "      <td>0.1238</td>\n",
       "      <td>0.1866</td>\n",
       "      <td>0.2416</td>\n",
       "      <td>0.1860</td>\n",
       "      <td>0.2750</td>\n",
       "      <td>0.08902</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>84300903</td>\n",
       "      <td>M</td>\n",
       "      <td>19.69</td>\n",
       "      <td>21.25</td>\n",
       "      <td>130.00</td>\n",
       "      <td>1203.0</td>\n",
       "      <td>0.10960</td>\n",
       "      <td>0.15990</td>\n",
       "      <td>0.1974</td>\n",
       "      <td>0.12790</td>\n",
       "      <td>...</td>\n",
       "      <td>25.53</td>\n",
       "      <td>152.50</td>\n",
       "      <td>1709.0</td>\n",
       "      <td>0.1444</td>\n",
       "      <td>0.4245</td>\n",
       "      <td>0.4504</td>\n",
       "      <td>0.2430</td>\n",
       "      <td>0.3613</td>\n",
       "      <td>0.08758</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>84348301</td>\n",
       "      <td>M</td>\n",
       "      <td>11.42</td>\n",
       "      <td>20.38</td>\n",
       "      <td>77.58</td>\n",
       "      <td>386.1</td>\n",
       "      <td>0.14250</td>\n",
       "      <td>0.28390</td>\n",
       "      <td>0.2414</td>\n",
       "      <td>0.10520</td>\n",
       "      <td>...</td>\n",
       "      <td>26.50</td>\n",
       "      <td>98.87</td>\n",
       "      <td>567.7</td>\n",
       "      <td>0.2098</td>\n",
       "      <td>0.8663</td>\n",
       "      <td>0.6869</td>\n",
       "      <td>0.2575</td>\n",
       "      <td>0.6638</td>\n",
       "      <td>0.17300</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>84358402</td>\n",
       "      <td>M</td>\n",
       "      <td>20.29</td>\n",
       "      <td>14.34</td>\n",
       "      <td>135.10</td>\n",
       "      <td>1297.0</td>\n",
       "      <td>0.10030</td>\n",
       "      <td>0.13280</td>\n",
       "      <td>0.1980</td>\n",
       "      <td>0.10430</td>\n",
       "      <td>...</td>\n",
       "      <td>16.67</td>\n",
       "      <td>152.20</td>\n",
       "      <td>1575.0</td>\n",
       "      <td>0.1374</td>\n",
       "      <td>0.2050</td>\n",
       "      <td>0.4000</td>\n",
       "      <td>0.1625</td>\n",
       "      <td>0.2364</td>\n",
       "      <td>0.07678</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 33 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         id diagnosis  radius_mean  texture_mean  perimeter_mean  area_mean  \\\n",
       "0    842302         M        17.99         10.38          122.80     1001.0   \n",
       "1    842517         M        20.57         17.77          132.90     1326.0   \n",
       "2  84300903         M        19.69         21.25          130.00     1203.0   \n",
       "3  84348301         M        11.42         20.38           77.58      386.1   \n",
       "4  84358402         M        20.29         14.34          135.10     1297.0   \n",
       "\n",
       "   smoothness_mean  compactness_mean  concavity_mean  concave points_mean  \\\n",
       "0          0.11840           0.27760          0.3001              0.14710   \n",
       "1          0.08474           0.07864          0.0869              0.07017   \n",
       "2          0.10960           0.15990          0.1974              0.12790   \n",
       "3          0.14250           0.28390          0.2414              0.10520   \n",
       "4          0.10030           0.13280          0.1980              0.10430   \n",
       "\n",
       "   ...  texture_worst  perimeter_worst  area_worst  smoothness_worst  \\\n",
       "0  ...          17.33           184.60      2019.0            0.1622   \n",
       "1  ...          23.41           158.80      1956.0            0.1238   \n",
       "2  ...          25.53           152.50      1709.0            0.1444   \n",
       "3  ...          26.50            98.87       567.7            0.2098   \n",
       "4  ...          16.67           152.20      1575.0            0.1374   \n",
       "\n",
       "   compactness_worst  concavity_worst  concave points_worst  symmetry_worst  \\\n",
       "0             0.6656           0.7119                0.2654          0.4601   \n",
       "1             0.1866           0.2416                0.1860          0.2750   \n",
       "2             0.4245           0.4504                0.2430          0.3613   \n",
       "3             0.8663           0.6869                0.2575          0.6638   \n",
       "4             0.2050           0.4000                0.1625          0.2364   \n",
       "\n",
       "   fractal_dimension_worst  Unnamed: 32  \n",
       "0                  0.11890          NaN  \n",
       "1                  0.08902          NaN  \n",
       "2                  0.08758          NaN  \n",
       "3                  0.17300          NaN  \n",
       "4                  0.07678          NaN  \n",
       "\n",
       "[5 rows x 33 columns]"
      ]
     },
     "execution_count": 186,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cancer.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 187,
   "metadata": {},
   "outputs": [],
   "source": [
    "cancer.drop(columns = ['Unnamed: 32', 'id'], inplace = True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 188,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "B    357\n",
       "M    212\n",
       "Name: diagnosis, dtype: int64"
      ]
     },
     "execution_count": 188,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cancer['diagnosis'].value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 189,
   "metadata": {},
   "outputs": [],
   "source": [
    "cancer['diagnosis'].replace('B', 1, inplace = True)\n",
    "cancer['diagnosis'].replace('M', 2, inplace = True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 190,
   "metadata": {},
   "outputs": [],
   "source": [
    "cancer = cancer.astype(float).values.tolist()\n",
    "random.shuffle(ads)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 191,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_size = 0.2\n",
    "train_label = {1:[],2:[]}\n",
    "test_label = {1:[],2:[]}\n",
    "train_data = cancer[:-int(test_size*len(cancer))]\n",
    "test_data = cancer[-int(test_size * len(cancer)):]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 192,
   "metadata": {},
   "outputs": [],
   "source": [
    "for i in train_data:\n",
    "    train_label[i[0]].append(i[1:])\n",
    "    \n",
    "for i in test_data:\n",
    "    test_label[i[0]].append(i[1:])\n",
    "        \n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 193,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy:  0.9292035398230089\n"
     ]
    }
   ],
   "source": [
    "correct = 0\n",
    "total = 0\n",
    "for group in test_label:\n",
    "    for data in test_label[group]:\n",
    "        vote = k_nearest_neighbor(train_label, data, k = 9)\n",
    "        if group == vote:\n",
    "            correct +=1\n",
    "        total +=1\n",
    "        \n",
    "print('accuracy: ', correct/total)        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Iris dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 194,
   "metadata": {},
   "outputs": [],
   "source": [
    "iris = pd.read_csv('Iris.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 195,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Id</th>\n",
       "      <th>SepalLengthCm</th>\n",
       "      <th>SepalWidthCm</th>\n",
       "      <th>PetalLengthCm</th>\n",
       "      <th>PetalWidthCm</th>\n",
       "      <th>Species</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>5.1</td>\n",
       "      <td>3.5</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>Iris-setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>4.9</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>Iris-setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>4.7</td>\n",
       "      <td>3.2</td>\n",
       "      <td>1.3</td>\n",
       "      <td>0.2</td>\n",
       "      <td>Iris-setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>4.6</td>\n",
       "      <td>3.1</td>\n",
       "      <td>1.5</td>\n",
       "      <td>0.2</td>\n",
       "      <td>Iris-setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>5.0</td>\n",
       "      <td>3.6</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>Iris-setosa</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Id  SepalLengthCm  SepalWidthCm  PetalLengthCm  PetalWidthCm      Species\n",
       "0   1            5.1           3.5            1.4           0.2  Iris-setosa\n",
       "1   2            4.9           3.0            1.4           0.2  Iris-setosa\n",
       "2   3            4.7           3.2            1.3           0.2  Iris-setosa\n",
       "3   4            4.6           3.1            1.5           0.2  Iris-setosa\n",
       "4   5            5.0           3.6            1.4           0.2  Iris-setosa"
      ]
     },
     "execution_count": 195,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "iris.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 196,
   "metadata": {},
   "outputs": [],
   "source": [
    "iris.drop(columns = ['Id'], inplace = True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 197,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Iris-setosa        50\n",
       "Iris-versicolor    50\n",
       "Iris-virginica     50\n",
       "Name: Species, dtype: int64"
      ]
     },
     "execution_count": 197,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "iris['Species'].value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 198,
   "metadata": {},
   "outputs": [],
   "source": [
    "iris['Species'].replace('Iris-setosa',1, inplace = True)\n",
    "iris['Species'].replace('Iris-versicolor',2, inplace = True)\n",
    "iris['Species'].replace('Iris-virginica',3, inplace = True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 199,
   "metadata": {},
   "outputs": [],
   "source": [
    "iris = iris.astype(float).values.tolist()\n",
    "random.shuffle(iris)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 200,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_size = 0.3\n",
    "train_label = {1:[],2:[], 3:[]}\n",
    "test_label = {1:[],2:[], 3: []}\n",
    "train_data = iris[:-int(test_size*len(iris))] \n",
    "test_data = iris[-int(test_size * len(iris)):]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 201,
   "metadata": {},
   "outputs": [],
   "source": [
    "for i in train_data:\n",
    "    train_label[i[-1]].append(i[:-1]) \n",
    "    \n",
    "for i in test_data:\n",
    "    test_label[i[-1]].append(i[:-1])\n",
    "        \n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 203,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy:  0.9777777777777777\n"
     ]
    }
   ],
   "source": [
    "correct = 0\n",
    "total = 0\n",
    "for group in test_label:\n",
    "    for data in test_label[group]:\n",
    "        vote = k_nearest_neighbor(train_label, data, k =9 )\n",
    "        if group == vote:\n",
    "            correct +=1\n",
    "        total +=1\n",
    "        \n",
    "print('accuracy: ', correct/total)        "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}