nn_implementation.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h3 align = 'center'> Variables </h3>\n",
    "\n",
    "|Code Symbol | Math Symbol | Definition | Dimensions\n",
    "| :-: | :-: | :-: | :-: |\n",
    "|X|$$X$$|Input Data, each row is an example| (numExamples, inputLayerSize)|\n",
    "|y |$$y$$|target data|(numExamples, outputLayerSize)|"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let’s say you want to predict some output value y, given some input value X. For example, maybe you want to predict your score on a test based on how many hours you sleep and how many hours you study the night before. To use a machine learning approach, we first need some data. Let’s say for the last three tests, you recorded your number of hours of studying, your number of hours sleeping, and your score on the test. We'll use the programming language python to store our data in 2-dimensional numpy arrays."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "# X = (hours sleeping, hours studying), y = Score on test\n",
    "X = np.array(([3,5], [5,1], [10,2]), dtype=float)\n",
    "y = np.array(([75], [82], [93]), dtype=float)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[  3.,   5.],\n",
       "       [  5.,   1.],\n",
       "       [ 10.,   2.]])"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 75.],\n",
       "       [ 82.],\n",
       "       [ 93.]])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "X = X/np.amax(X, axis=0)\n",
    "y = y/100 #Max test score is 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.75],\n",
       "       [ 0.82],\n",
       "       [ 0.93]])"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.3,  1. ],\n",
       "       [ 0.5,  0.2],\n",
       "       [ 1. ,  0.4]])"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have some data, we’re going to use it to train a model to predict how you will do on your next test, based on how many hours you sleep and how many hours you study. This is called a supervised regression problem. It’s supervised because our examples have inputs and outputs. It’s a regression problem because we’re predicting your test score, which is a continuous output. If we we’re predicting your letter grade, this would be called a classification problem, and not a regression problem.\n",
    "\n",
    "\n",
    "Before we throw our data into the model, we need to account for the differences in the units of our data. Both of our inputs are in hours, but our output is a test score, scaled between 0 and 100. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can build our Neural Net. We know our network must have 2 inputs and 1 output, because these are the dimensions of our data. We’ll call our output y hat, because it’s an estimate of y, but not the same as y. Any layer between our input and output layer is called a hidden layer. Recently, researchers have built networks with many many hidden layers. These are known as a deep belief networks, giving rise to the term deep learning. Here, we’re going to use 1 hidden layer with 3 hidden units, but if we wanted to build a deep neural network, we would just stack a bunch of layers together."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h3 align = 'center'> Variables </h3>\n",
    "\n",
    "|Code Symbol | Math Symbol | Definition | Dimensions\n",
    "| :-: | :-: | :-: | :-: |\n",
    "|X|$$X$$|Input Data, each row in an example| (numExamples, inputLayerSize)|\n",
    "|y |$$y$$|target data|(numExamples, outputLayerSize)|\n",
    "|W1 | $$W^{(1)}$$ | Layer 1 weights | (inputLayerSize, hiddenLayerSize) |\n",
    "|W2 | $$W^{(2)}$$ | Layer 2 weights | (hiddenLayerSize, outputLayerSize) |\n",
    "|z2 | $$z^{(2)}$$ | Layer 2 activation | (numExamples, hiddenLayerSize) |\n",
    "|a2 | $$a^{(2)}$$ | Layer 2 activity | (numExamples, hiddenLayerSize) |\n",
    "|z3 | $$z^{(3)}$$ | Layer 3 activation | (numExamples, outputLayerSize) |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our network has 2 inputs, 3 hidden units, and 1 output. These are examples of hyperparameters. Hyperparameters are constants that establish the structure and behavior of a neural network, but are not updated as we train the network. Our learning algorithm is not capable of, for example, deciding that it needs another hidden unit, this is something that WE must decide on before training. What a neural network does learn are parameters, specifically the weights on the synapses."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We’ll take care of moving data through our network in a method called forward. Rather than pass inputs through the network one at a time, we’re going to use matrices to pass through multiple inputs at once. Doing this allows for big computational speedups, especially when using tools like MATLAB or Numpy. Our input data matrix, X, is of dimension 3 by 2, because we have 3, 2-dimensional examples. Our corresponding output data, y, is of dimension 3 by 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(3, 2) (3, 1)\n"
     ]
    }
   ],
   "source": [
    "print X.shape, y.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Each input value, or element in matrix X, needs to be multiplied by a corresponding weight and then added together with all the other results for each neuron. This is a complex operation, but if we take the three outputs we're looking for as a single row of a matrix, and place all our individual weights into a matrix of weights, we can create the exact behavior we need by multiplying our input data matrix by our weight matrix. Using matrix multiplication allows us to pass multiple inputs through at once by simply adding rows to the matrix X. From here on out, we'll refer to these matrics as X, W one, and z two, where z two the activity of our second layer. Notice that each entry in z is a sum of weighted inputs to each hidden neuron. Z is of size 3 by 3, one row for each example, and one column for each hidden unit. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now have our first official formula, $z^{(2)} = XW^{(1)}$. Matrix notation is really nice here, becuase it allows us to express the complex underlying process in a single line!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have the activities for our second layer, z two, we need to apply the activation function. We'll independently apply the function to each entry in matrix z using a python method for this called sigmoid, because we’re using a sigmoid as our activation function. Using numpy is really nice here, because we can pass in a scalar, vector, or matrix, Numpy will apply the activation function element-wise, and return a result of the same dimension as it was given."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def sigmoid(z):\n",
    "    #Apply sigmoid activation function to scalar, vector, or matrix\n",
    "    return 1/(1+np.exp(-z))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEACAYAAACj0I2EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl8VdXV//HPAkUKIqngCCKoSKn6EPSBR1FsrFVwqFZb\nRX9O1KlWEYci4khwqEp5qvigFi2WOpXWecShShyrYjXOKKAgoHUCtFJlSNbvjx1MiCE5hHNz7t33\n+3697uuec7Nzs5ZHVk7W2Xcfc3dERKTwtco6ABERSYcKuohIJFTQRUQioYIuIhIJFXQRkUiooIuI\nRKLJgm5mk8zsYzN7rZExV5vZTDOrNLPSdEMUEZEkkpyh/wkYtLovmtk+wNbu3hP4FfCHlGITEZE1\n0GRBd/dngEWNDDkQuKlm7AtARzPbJJ3wREQkqTR66F2AeXX2F9S8JiIiLSiNgm4NvKb1BEREWtg6\nKbzHfGCLOvtdgQ8bGmhmKvQiIs3g7g2dPK8iaUE3Gj4TB7gPOAX4q5ntDCx2948bCSrhjyw85eXl\nlJeXZx1GzsScX8y5QfHl5w6ffw7z58OCBd99/uQT+PTT8Fi2rPk/d731oH17aNcuPOpur+7Rti20\naRO+t+5zQ6+tfP7hD5us5UCCgm5mtwFlQCcz+wAYDbQB3N2vd/eHzGxfM5sFLAF+2ez/OgVuzpw5\nWYeQUzHnF3NuEGd+S5fC++/DrFnw0ENzWLgQZs8OjzlzwteTaNcONtooPDp3rn0uKYGOHVf/2GCD\nUHDzSZMF3d3/X4Ixw9IJR0RkVcuXw8yZ8MYbqz5mz4bq6tpx06ev+n0dO0LXrtCly3efN920tnC3\na9ey+eRSGj10qTF06NCsQ8ipmPOLOTconPyqqmDGjFCcX3wxPL/6aijq9bVuDVttBVtvDW3bDmX3\n3cP21ltDjx7QoUPLx581a8metpl5zD10EVkzS5eGwl1RER4vvghfffXdcT16wPbbr/ro1Sv/Wh65\nYmapXhSVBCoqKigrK8s6jJyJOb+Yc4P8yc8dXnkFpk6FadPguefg669XHdOtG/TvD/36hecddwz9\n6sbkS35ZU0EXkZxasgQefxweeAAefBA+rDepefvtoawM9tgDBgwI/W1pHrVcRCR133wTzsL/8pdQ\nyOuehXfpAvvtB3vtBbvvDhtvnF2chUItFxFpUe6hD/7nP8Pdd8OXX9Z+rV8/+OlPw6NPH7Bk06pl\nDWk99BRVVFRkHUJOxZxfzLlBbvP75BMYOzZcpPzxj0NB//LL0PseOxbmzg0XOy+4AEpLc1PMYz9+\nSekMXUSa5eWXYdw4uOOO2mmFXbrAscfCEUeEAi8tSz10EUnMHR5+GH73uzBLBaBVK9h/fzjhBBg8\nGNbRaWLq1EMXkdS4wz33wOjR8Prr4bUOHeDEE2H48DDVULKnHnqKYu/jxZxfzLlB8/NzD7NV+vWD\ngw8OxXzzzUNvfN680HLJh2Ie+/FLSmfoItKgl1+G00+Hp58O+5tuCuefD8cfXzyf0Cw06qGLyCo+\n+QTOOw8mTQpn6J07w6hR8Otfx7WQVSFRD11E1khVFVxzDVx4IXzxRbi4edppYbphx45ZRydJqIee\notj7eDHnF3Nu0HR+b70Fu+0WCvgXX8A++4QlaseNK4xiHvvxS0oFXaSILV8OF18MffvC88+HC573\n3AMPPaR55IVIPXSRIjV7Nhx+eO2NIU44IcwvL4Qz8mKjHrqIrNbNN8PJJ4e1x7fYAiZPDh/bl8Km\nlkuKYu/jxZxfzLlBbX5LlsBRR8HRR4difsgh4Y5AhV7MYz9+SekMXaRIvP8+/Oxn8NprYfrh1VeH\ndVe08mE81EMXKQKPPQaHHQYLF0LPnuHC5w9/mHVUklTSHrpaLiIRc4crrwyLZi1cGBbRevFFFfNY\nqaCnKPY+Xsz5xZhbVRWccQaceSZUV1dwwQVw771QUpJ1ZOmL8fg1h3roIhH6+utw8fPOO6FNGxg5\nEi66KOuoJNfUQxeJzMKF4VZvzz0X5pTfc0+4CbMULs1DFylCn34Ke+8NlZVhfvnUqbDddllHJS1F\nPfQUxd7Hizm/GHL7179gjz1CMe/ZM5yhryzmMeTXmNjzS0oFXSQCCxaEtsqbb4YZLE8+CV27Zh2V\ntDT10EUK3Ecfwe67w6xZ0KdPmHO+0UZZRyVpUg9dpAgsXBh65rNmhRUT//532HDDrKOSrKjlkqLY\n+3gx51eIuf3737XrlvfuDY8+uvpiXoj5rYnY80tKBV2kAH3zDRx4YPjUZ48eoc3SuXPWUUnW1EMX\nKTBVVXDooXDXXbDZZvDMM7DVVllHJbmktVxEIjVyZCjmHTuGNouKuaykgp6i2Pt4MedXKLldey38\n/vfhBs533QXbb5/s+wolv+aKPb+kVNBFCsSDD8Kpp4btP/6x8G9KIelL1EM3s8HAVYRfAJPc/Yp6\nX98C+DNQUjPmHHef2sD7qIcu0gyvvw677BLuOHTBBVpoq9gk7aE3WdDNrBXwLrAn8CEwHTjM3WfU\nGTMReNndJ5pZb+Ahd+/RwHupoIusoUWLoF+/2ps633qr7jJUbNK8KNofmOnuc919OTAFOLDemGpg\ng5rtEmDBmgQbi9j7eDHnl6+5VVXBEUeEYt63b2i1NKeY52t+aYk9v6SSfFK0CzCvzv58QpGvawzw\nqJkNB9oBP0knPJHiVl4eVkzs1ClcBG3XLuuIJJ8lKegNnQ/U75scDvzJ3a80s52BW4AGF+0cOnQo\n3bt3B6CkpITS0lLKahZrXvlbtlD3V76WL/Eov+T7ZWVleRUPwCWXVHDJJdCqVRlTpsCcORXMmRNP\nfrEfv7XZr6ioYPLkyQDf1sskkvTQdwbK3X1wzf4owOteGDWzN4BB7r6gZn828D/u/lm991IPXSSB\n99+H0lL48kv43e9gxIisI5IspdlDnw5sY2Zbmlkb4DDgvnpj5lLTZqm5KLpe/WJeDFb+ho1VzPnl\nU27LlsFhh4ViftBB8JvfrP175lN+uRB7fkk1WdDdvQoYBjwKvAlMcfe3zWyMme1fM2wEcIKZVQK3\nAsfkKmCR2J1/flijpVs3mDRJM1okOa3lIpJHpk6FffeF1q3hqadgwICsI5J8oLVcRArMRx/BMTV/\n2158sYq5rDkV9BTF3seLOb+sc3OHY48NN3n+yU/g7LPTff+s88u12PNLSgVdJA9cfz08/HC4QcVN\nN0Er/cuUZlAPXSRjs2eHe4EuWQJTpsCQIVlHJPlGPXSRAlBVBUOHhmI+ZIiKuawdFfQUxd7Hizm/\nrHL7/e/DHYc22wyuuSZ3PyfmYwfx55eUCrpIRt58M8w5h7DoVqdO2cYjhU89dJEMVFXBbrvB88/D\n8cfDDTdkHZHkM/XQRfLYddeFYr755jBuXNbRSCxU0FMUex8v5vxaMrd58+Ccc8L2hAnhZs+5FvOx\ng/jzS0oFXaQFucPJJ8NXX4WFtw46KOuIJCbqoYu0oL/9LUxN3GADePvt0HIRaYp66CJ5ZtEiGD48\nbI8dq2Iu6VNBT1HsfbyY82uJ3EaNgo8/hoED4YQTcv7jVhHzsYP480tKBV2kBbz0UpiauM468Ic/\naK0WyQ310EVyrLoadtkl3LRixIhwSzmRNZG0h66CLpJjN94Ixx0XeuYzZkCHDllHJIVGF0UzEHsf\nL+b8cpXbokW1a5uPG5ddMY/52EH8+SWlgi6SQxdeCJ99Bj/6Ubjxs0guqeUikiOVlbDTTuEmz6+8\nAjvskHVEUqjUchHJkDucemq4IDpsmIq5tAwV9BTF3seLOb+0c7v99rDO+cYbw5gxqb51s8R87CD+\n/JJSQRdJ2Tff1F4IveSSlll8SwTUQxdJ3dixoaBvv33oo7dunXVEUug0D10kA59+CttsA19+CY88\nAnvvnXVEEgNdFM1A7H28mPNLK7fy8lDM99knv4p5zMcO4s8vKRV0kZS89RZMnBjWadHH+yULarmI\npGS//eChh+Ckk8It5kTSoh66SAt67LHQYunQAWbNCtMVRdKiHnoGYu/jxZzf2uRWXQ1nnRW2zz03\nP4t5zMcO4s8vKRV0kbU0ZQq8+ip07QqnnZZ1NFLM1HIRWQvLlkHv3vDeezBpEhx7bNYRSYzUchFp\nATfcEIr5D34ARx+ddTRS7FTQUxR7Hy/m/JqT21dfwcUXh+1LLw23l8tXMR87iD+/pFTQRZpp/Phw\n0+f+/eGgg7KORiRhD93MBgNXEX4BTHL3KxoYcygwGqgGXnX3IxsYox66ROHzz2GrrcKnQp94AvbY\nI+uIJGZJe+hN/pFoZq2ACcCewIfAdDO7191n1BmzDXA2sIu7f2lmnZsfukj+u+yyUMz33lvFXPJH\nkpZLf2Cmu8919+XAFODAemNOAK5x9y8B3P2zdMMsDLH38WLOb01ymzcPJkwI25ddlpt40hbzsYP4\n80sqSUHvAsyrsz+/5rW6tgV6mdkzZvacmQ1KK0CRfFNeDkuXwpAhsOOOWUcjUqvJHrqZ/QLY291P\nrNk/Eujn7qfVGXM/sAw4BOgGPA1st/KMvc449dCloL39dljn3Cxs9+yZdURSDFLroRPOyLvV2e9K\n6KXXH/MPd68G5pjZO0BP4J/132zo0KF0794dgJKSEkpLSykrKwNq/2zSvvbzdf+ii6C6uowTT4QF\nCypYsCC/4tN+HPsVFRVMnjwZ4Nt6mYi7N/oAWgOzgC2BNkAl0LvemEHA5JrtzsBc4PsNvJfHbNq0\naVmHkFMx55ckt9dfdzdzb9PG/YMPch9TmmI+du7x51dTO5us10320N29ChgGPAq8CUxx97fNbIyZ\n7V8z5hHgczN7E3gcGOHui5L/WhHJf2PGgDuccAJssUXW0Yh8l9ZyEUngtdegTx9Ybz2YPRu61J8W\nIJJDWstFJEVjxoTnX/1KxVzylwp6ilZe1IhVzPk1lltlJdx1F7RtC6NGtVxMaYr52EH8+SWlgi7S\nhPLy8HzSSbDZZpmGItIo9dBFGvHPf8J//zd873thmdxNN806IilG6qGLpGDl2fnJJ6uYS/5TQU9R\n7H28mPNrKLfp0+GBB6BdOxg5suVjSlPMxw7izy8pFXSR1Vh5dj5sWH7e+FmkPvXQRRrw/POwyy7Q\nvj3MmQOdtSC0ZEg9dJG1sPLsfPhwFXMpHCroKYq9jxdzfnVze+45eOQR6NABfvOb7GJKU8zHDuLP\nLykVdJF6Ro8Oz8OHQ6dO2cYisibUQxep4+mnYffdYYMN4P33YcMNs45IRD10kWZZeXZ++ukq5lJ4\nVNBTFHsfL+b8KioqePJJmDYNOnaEM87IOqJ0xXzsIP78klJBFyGsc37hhWH7zDOhpCTbeESaQz10\nEeCJJ2DPPUMhnzMnnKWL5Av10EUScq/tnY8YoWIuhUsFPUWx9/Fize/xx+GZZyrYcEM49dSso8mN\nWI/dSrHnl5QKuhS1ur3zESPCdEWRQqUeuhS1Rx6BwYPDx/vfey98OlQk36iHLtKEur3zs85SMZfC\np4Keotj7eLHlN3UqvPACbLQR7LBDRdbh5FRsx66+2PNLSgVdilLds/Ozzw63mBMpdOqhS1G6/344\n4ADYZJPQO2/XLuuIRFZPPXSR1ah/dq5iLrFQQU9R7H28WPK791545RXYbDM46aTwWiy5rY7yKw4q\n6FJUqqtrz87PPVe9c4mLeuhSVO64Aw45BLp2hZkzoW3brCMSaZp66CL1VFXVnp2fd56KucRHBT1F\nsffxCj2/22+Ht96Cbt3g2GNX/Vqh59YU5VccVNClKFRVQXl52D7/fGjTJtNwRHJCPXQpCrfcAkcd\nBT16wDvvwLrrZh2RSHLqoYvUWLECLroobF9wgYq5xEsFPUWx9/EKNb9bbw0zWrbeOpylN6RQc0tK\n+RUHFXSJ2vLltWfno0fDOutkG49ILqmHLlGbNAmOPx569YI33lBBl8KUag/dzAab2Qwze9fMzm5k\n3C/MrNrMdlyTYEVyYdkyuPjisK2zcykGTRZ0M2sFTAAGAdsBh5vZDxoYtz5wKvB82kEWitj7eIWW\n3403wty50Ls3HHpo42MLLbc1pfyKQ5Iz9P7ATHef6+7LgSnAgQ2Muxi4AliaYnwizfL117Vn5+Xl\n0Lp1puGItIgme+hm9nNgkLufWLN/JNDf3YfXGVMKnOfuh5jZNOA37v5yA++lHrq0iHHjwm3l+vaF\nl16CVrr8LwUsaQ89SVexoTf5tiqbmQFXAsc08T0iLeKLL+Cyy8L2pZeqmEvxSFLQ5wPd6ux3BT6s\ns9+B0FuvqCnumwL3mtkBDZ2lDx06lO7duwNQUlJCaWkpZWVlQG0frFD3r7rqqqjyKdT8nniijIUL\nw31CwwJcTX9/3R5s1vHnYl/5FdZ+RUUFkydPBvi2Xibi7o0+gNbALGBLoA1QCfRuZPw0oO9qvuYx\nmzZtWtYh5FQh5Pfxx+7t27uD+zPPJP++QshtbSi/wlZTO5us14nmoZvZYGA84SLqJHe/3MzGANPd\n/YF6Y58ARrh66JKB00+H8eNhv/3ggQeaHi9SCJL20PXBIonGBx9Az55h/nllJfTpk3VEIunQ4lwZ\nqNvHi1G+5zdmTCjmhx225sU833NbW8qvOKigSxRmzIDJk8N885Vrt4gUG7VcJAqHHBLuF3riiTBx\nYtbRiKRLPXQpGv/4BwwYEO4ROnNmuAG0SEzUQ89A7H28fMzPHUaMCNtnntn8Yp6PuaVJ+RUHFXQp\naHffDc89BxttBGevdh1QkeKglosUrGXLYLvtYNYsmDABTjkl64hEckMtF4nexImhmG+7bbgYKlLs\nVNBTFHsfL5/y++KLMO8c4Ior1v7Gz/mUWy4ov+Kggi4F6bLL4PPPYeBAOLCh1flFipB66FJwPvgg\ntFmWLoUXXoD+/bOOSCS31EOXaJ1zTijmhx2mYi5Slwp6imLv4+VDfs8+C7fdBuutB7/9bXrvmw+5\n5ZLyKw4q6FIwqqrg1FPD9siR0KNHtvGI5Bv10KVg3HBDmJ7YtWtYjKt9+6wjEmkZWstForJ4cVjr\n/LPP4C9/Cf1zkWKhi6IZiL2Pl2V+5eWhmA8cCEOGpP/+OnaFLfb8klJBl7z31lvho/2tWsHVV4M1\neZ4iUpzUcpG85g577QWPPw4nnQTXXZd1RCItTz10icJtt8ERR8CGG8I770DnzllHJNLy1EPPQOx9\nvJbOb9EiOOOMsD12bG6LuY5dYYs9v6RU0CVvnXcefPIJ7LYb/PKXWUcjkv/UcpG89MILsMsu4abP\nlZVh3XORYqWWixSsFSvgV7+qvb2cirlIMiroKYq9j9dS+V19Nbz6KnTvDhdc0CI/UseuwMWeX1Iq\n6JJXZs2C888P2xMmQLt22cYjUkjUQ5e8UV0Ne+wBTz0VpireckvWEYnkB/XQpeBcd10o5ptsAuPH\nZx2NSOFRQU9R7H28XOb3/vtw9tlh+9proVOnnP2oBunYFbbY80tKBV0y5w4nnABLlsChh8LBB2cd\nkUhhUg9dMjdxYlinpXPnsBDXRhtlHZFIftFaLlIQ3n0X+vaF//wH/vrXcIYuIqvSRdEMxN7HSzu/\n5cvDbJb//Cc8Z1nMdewKW+z5JaWCLpkpL4eXXoItt4Rrrsk6GpHCp5aLZOKpp6CsLNys4sknwwJc\nItIwtVwkby1eDEcdFWa3nHOOirlIWhIVdDMbbGYzzOxdMzu7ga+fYWZvmlmlmT1mZlukH2r+i72P\nl0Z+7nDMMfDBB9CvH4wevfZxpUHHrrDFnl9STRZ0M2sFTAAGAdsBh5vZD+oNexnYyd1LgTuB36Ud\nqMRh3Di47z4oKQmzWtZdN+uIROLRZA/dzHYGRrv7PjX7owB39ytWM74U+D93H9jA19RDL2JPPx3W\naqmqgnvvhQMOyDoikcKQZg+9CzCvzv78mtdW5zhgaoL3lSLy8ccwZEgo5iNHqpiL5MI6CcY09Fuh\nwdNsMzsS2An40erebOjQoXTv3h2AkpISSktLKSsrA2r7YIW6f9VVV0WVT1r57bprGYcfDh99VMEO\nO8Cll+ZHPnX36/Zg8yEe5Vfc+VVUVDB58mSAb+tlIu7e6APYGXi4zv4o4OwGxv0EeBPo1Mh7ecym\nTZuWdQg51dz8TjnFHdw32cR9wYJ0Y0qLjl1hiz2/mtrZZL1O0kNvDbwD7Al8BLwIHO7ub9cZ0xe4\nHRjk7rMbeS9v6udJXFau09KmDVRUhPuEisiaSa2H7u5VwDDgUcIZ+BR3f9vMxpjZ/jXDxgLtgdvN\n7BUzu2ctYpdIVFTAsGFh+4YbVMxFci3RPHR3f9jde7l7T3e/vOa10e7+QM32Xu6+mbvv6O593f1n\nuQw6X9Xt48VoTfJ77z34xS/CDZ9HjICjj85dXGnQsStsseeXlD4pKqn79FMYPBg+/xz23Rcuvzzr\niESKg9ZykVQtWRLmmk+fDqWlYZ2WDTbIOiqRwqa1XKTFLV8elsCdPh26d4eHHlIxF2lJKugpir2P\n11h+1dVw4omhiHfqBI88Aptt1nKxra1iPnYxiD2/pFTQZa25w6mnwuTJ0K4dPPggbLtt1lGJFB/1\n0GWtuMMZZ8D48WGu+X33waBBWUclEhf10CXn3MO6LOPHh1UT775bxVwkSyroKYq9j1c3P3cYNSos\nh7vOOnDHHWGKYqEqpmMXo9jzSyrJ4lwiq6iqgpNPhuuvh9atw7rmWj1RJHvqocsaWbYs3D7ub3+D\ntm3h9tth//2b/j4Rab6kPXSdoUtiS5bAz38epiRusAHcfz/svnvWUYnISuqhpyjmPt78+dCnTwWP\nPAIbbQTTpsVVzGM+dqD8ioUKujTppZegf3+YPRu22QaeeQZ23DHrqESkPvXQpVF33BFWSvz6aygr\nC/udOmUdlUhx0Tx0WSvLl4dlbw85JBTz444LvXMVc5H8pYKeolj6ePPnhxUT//d/wxzzK68MN6h4\n7rmKjCPLnViO3eoov+KgWS6yiqlTQ4vls8+gS5cwPXHAgKyjEpEk1EMXAL76KrRYJk4M+3vvDbfc\nEma0iEi21EOXxJ59Fvr0CcW8TRu44oqwDK6KuUhhUUFPUaH18RYvhlNOgYEDwz1A+/QJUxRHjgwf\n6a+v0PJbEzHnBsqvWKigFyF3mDIFeveGa68Nxfvcc+HFF2GHHbKOTkSaSz30IvPKK3DWWfD442F/\nwAD4wx9UyEXymXrosop58+CYY2CnnUIx//73w1TEp59WMReJhQp6ivKxj/evf4Uz8m23hZtuCvPK\nzzgDZs2C44+HVmvwf0A+5peWmHMD5VcsNA89UgsWwNixYc3yb74Jrw0ZAr/9LWy1VbaxiUhuqIce\nmddeg6uvhptvDmuXAxx4IFxwQWi3iEjh0XroRWTFirA2+fjx8OST4TWzsA7L+efDf/1XtvGJSMtQ\nDz1FLd3He+edMN2we3c4+OBQzDt0gOHDYcaM8LH9NIt5zH3KmHMD5VcsdIZeYD79FO66C/78Z/jH\nP2pf79kThg2DoUPD3YREpPioh14AFiyAu++GO++Ep56C6urw+vrrhwudQ4fCrruGNouIxEc99AK2\nfDm88AI8+mhYg/zFF2u/tu66YeGsI46Agw6C9u2zi1NE8ot66Clqbh+vqgoqK2HCBPjZz8JNJAYO\nhIsvDsW8bdtQvG++GT75JCxxe+SRLV/MY+5TxpwbKL9ioTP0DCxeHBbBevbZ8Hj+efj3v1cd06sX\nDBoUzsbLynQmLiJNUw89h9zD3X9eeSWcgVdWhu05c747tkePsK7Kj34UiviWW7Z4uCKSp9RDb0FL\nlsDMmWEa4YwZ4fmdd+Ddd8ONI+pr2zZMJxwwIFzMHDAANt+85eMWkbgkKuhmNhi4itBzn+TuV9T7\nehvgJmAn4DNgiLt/kHKsmVixIvStP/oI5s4Njw8+WHX7s89Wjq4Aylb5/s6dwzrjpaXQt2947tUr\nrKlSaCoqKigrK8s6jJyIOTdQfsWiybJiZq2ACcCewIfAdDO7191n1Bl2HLDQ3Xua2RBgLHBYLgJe\nG+6hV714MSxaFB4rtz/+uOFHbbFevTZtYOutoXXrSvbbr4xevfj20alT7vNqKZWVldH+o4k5N1B+\nxSLJeWJ/YKa7zwUwsynAgUDdgn4gMLpm+w7CL4Bmqa6GpUvDY9my2u2lS8MiU0uWhDbGkiW1j9Xt\nf/VVbcFevDg8qqrWLB6zcCu2TTcNfe0tt4Ru3Vbd3mSTsGphefliysubm3n+W7x4cdYh5EzMuYHy\nKxZJCnoXYF6d/fmEIt/gGHevMrPFZrahuy+s/2b9+q1apOsX7hUrmptKMu3bQ0lJWA985fP3vw8b\nbxwKc93HppuGlklDt2MTEck3SQp6Q1dW609VqT/GGhgDhOl6TVlvvdDGWG+97z7WXz8U5fbtk23X\nLdodO4b3zZU5DU1fiUjM+cWcGyi/YtHktEUz2xkod/fBNfujAK97YdTMptaMecHMWgMfufvGDbxX\n8cxZFBFJUVrTFqcD25jZlsBHhIudh9cbcz9wDPACcAjwRHMDEhGR5mmyoNf0xIcBj1I7bfFtMxsD\nTHf3B4BJwM1mNhP4nDyc4SIiErsW/aSoiIjkTiaLc5nZqWY2w8xeN7PLs4gh18xshJlVm9mGWceS\nFjMba2Zvm1mlmd1pZlGsvG5mg2v+f3zXzM7OOp40mVlXM3vCzN6q+fc2POuY0mZmrczsZTO7L+tY\n0mZmHc3s9pp/d2+a2f80Nr7FC7qZlQE/BbZ39x2AcS0dQ66ZWVfgJ8DcrGNJ2aPAdu5eCswEzsk4\nnrVW54Nzg4DtgMPN7AfZRpWqFcCZ7v5DYBfglMjyAzgNeCvrIHJkPPCQu/cG+gBvNzY4izP0XwOX\nu/sKAHdP8FnMgnMlcFbWQaTN3f/u7jW31+B5oGuW8aTk2w/OuftyYOUH56Lg7v9y98qa7a8IBaFL\ntlGlp+bkaV/gj1nHkjYz6wAMdPc/Abj7Cnf/srHvyaKgbwvsbmbPm9k0M/vvDGLIGTP7KTDP3V/P\nOpYcOxaYmnUQKWjog3PRFLy6zKw7UEqYjRaLlSdPMV4M3Ar4zMz+VNNSut7MvtfYN+RkiSgzewzY\npO5LhP/3UA1oAAABxklEQVTg59f8zBJ339nM+gF/qwm8YDSR37nAXvW+VjAaye08d7+/Zsx5wHJ3\nvy2DENOW5INzBc/M1icsy3FazZl6wTOz/YCP3b2yppVbUP/WElgH2BE4xd1fMrOrgFHULrPS4Dek\nzt33Wt3XzOwk4K6acdNrLhx2cvfPcxFLLqwuPzPbHugOvGpmRmhJ/NPM+rv7Jy0YYrM1duwAzOwY\nwp+4P26ZiHJuPtCtzn5XwiJ00TCzdQjF/GZ3vzfreFK0K3CAme0LfA/oYGY3ufvRGceVlvmEv/ZX\nfr7+DqDRi/ZZtFzuIazciJltC6xbSMW8Me7+hrtv6u5buXsPwgHpWyjFvCk1yyiPBA5w96VZx5OS\nbz84V7MM9GFAbLMlbgTecvfxWQeSJnc/1927uftWhOP2RETFHHf/GJhXUych1M1GL/5msSr3n4Ab\nzex1YCkQzQFogBPXn4H/B7QBHgt/gPC8u5+cbUhrZ3UfnMs4rNSY2a7AEcDrZvYK4f/Jc9394Wwj\nk4SGA7ea2brAe8AvGxusDxaJiEQikw8WiYhI+lTQRUQioYIuIhIJFXQRkUiooIuIREIFXUQkEiro\nIiKRUEEXEYnE/wcoNKmyVbS15QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fea6eede3d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "testInput = np.arange(-6,6,0.01)\n",
    "plt.plot(testInput, sigmoid(testInput), linewidth= 2)\n",
    "plt.grid(1)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.7310585786300049"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sigmoid(1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.26894142,  0.5       ,  0.73105858])"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sigmoid(np.array([-1,0,1]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " We now have our second formula for forward propagation, using f to denote our activation function, we can write that a two, our second layer activity, is equal to f of z two. a two will be a matrix of the same size as z two, 3 by 3."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "a^{(2)} = f(z^{(2)}) \\tag{2}\\\\\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can initialize the network we built last time and pass in our normalized data, X, using our forward method, and have a look at our estimate of y, yHat."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class Neural_Network(object):\n",
    "    def __init__(self):        \n",
    "        #Define Hyperparameters\n",
    "        self.inputLayerSize = 2\n",
    "        self.outputLayerSize = 1\n",
    "        self.hiddenLayerSize = 3\n",
    "\n",
    "        #Weights (parameters)\n",
    "        self.W1 = np.random.randn(self.inputLayerSize,self.hiddenLayerSize)\n",
    "        self.W2 = np.random.randn(self.hiddenLayerSize,self.outputLayerSize)\n",
    "\n",
    "    def forward(self, X):\n",
    "        #Propogate inputs though network\n",
    "        self.z2 = np.dot(X, self.W1)\n",
    "        self.a2 = self.sigmoid(self.z2)\n",
    "        self.z3 = np.dot(self.a2, self.W2)\n",
    "        yHat = self.sigmoid(self.z3) \n",
    "        return yHat\n",
    "    \n",
    "    def sigmoid(self, z):\n",
    "        #Apply sigmoid activation function to scalar, vector, or matrix\n",
    "        return 1/(1+np.exp(-z))\n",
    "\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "NN = Neural_Network()\n",
    "\n",
    "yHat = NN.forward(X)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.69405427],\n",
       "       [ 0.69368203],\n",
       "       [ 0.73196479]])"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "yHat"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.75],\n",
       "       [ 0.82],\n",
       "       [ 0.93]])"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Right now our predictions are pretty inaccurate. To improve our model, we first need to quantify exactly how wrong our predictions are. We'll do this with a cost function. A cost function allows us to express exactly how wrong or \"costly\" our models is, given our examples."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "J = \\sum \\frac{1}{2}(y-\\hat{y})^2 \\tag{5}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "OUR cost is a function of two things, our examples, and the weights on our synapses. We don't have much control of our data, so we'll minimize our cost by changing the weights."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's return to the 1-dimensional case and see if we can be more clever. Let's evaluate our cost function for a specific value of w. If w is 1.1 for example, we can run our cost function, and see that J is 2.8. Now we haven't learned much yet, but let's try to add a little information to what we already know. What if we could figure out which way was downhill? If we could, we would know whether to make W smaller or larger do decrease the cost. We could test the cost function immediately to the left and right of our test point and see which is smaller. This is called numerical estimation, and is sometimes a good approach, but for us, there's a faster way. Let's look at our equations so far."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "z^{(2)} = XW^{(1)} \\tag{1}\\\\\n",
    "$$\n",
    "$$\n",
    "a^{(2)} = f(z^{(2)}) \\tag{2}\\\\\n",
    "$$\n",
    "$$\n",
    "z^{(3)} = a^{(2)}W^{(2)} \\tag{3}\\\\\n",
    "$$\n",
    "$$\n",
    "\\hat{y} = f(z^{(3)}) \\tag{4}\\\\\n",
    "$$\n",
    "$$\n",
    "J = \\sum \\frac{1}{2}(y-\\hat{y})^2 \\tag{5}\\\\\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have 5 equations, but we can really think of them as one big equation.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And since we have one big equation that uniquely determines our cost, J, from X, y, W1, and W2, we can use our good friend calculus to find what we're looking for. We want to know \"which way is downhill\", that is, what is the rate of change of J with respect to W, also known as the derivative. And in this case, since we’re just considering one weight at a time, the partial derivative. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can derive an expression for dJdW, that will give us the rate of change of J with respect to W, for any value of W! If dJdW is positive, then the cost function is going uphill. If dJdW is negative the cost function is going downhill. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This method is known as gradient descent, and although it may not seem so impressive in one dimension, it is capable of incredible speedups in higher dimensions. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h3 align = 'center'> Variables </h3>\n",
    "\n",
    "|Code Symbol | Math Symbol | Definition | Dimensions\n",
    "| :-: | :-: | :-: | :-: |\n",
    "|X|$$X$$|Input Data, each row in an example| (numExamples, inputLayerSize)|\n",
    "|y |$$y$$|target data|(numExamples, outputLayerSize)|\n",
    "|W1 | $$W^{(1)}$$ | Layer 1 weights | (inputLayerSize, hiddenLayerSize) |\n",
    "|W2 | $$W^{(2)}$$ | Layer 2 weights | (hiddenLayerSize, outputLayerSize) |\n",
    "|z2 | $$z^{(2)}$$ | Layer 2 activation | (numExamples, hiddenLayerSize) |\n",
    "|a2 | $$a^{(2)}$$ | Layer 2 activity | (numExamples, hiddenLayerSize) |\n",
    "|z3 | $$z^{(3)}$$ | Layer 3 activation | (numExamples, outputLayerSize) |\n",
    "|J | $$J$$ | Cost | (1, outputLayerSize) |\n",
    "|dJdz3 | $$\\frac{\\partial J}{\\partial z^{(3)} } = \\delta^{(3)}$$ | Partial derivative of cost with respect to $z^{(3)}$ | (numExamples,outputLayerSize)|\n",
    "|dJdW2|$$\\frac{\\partial J}{\\partial W^{(2)}}$$|Partial derivative of cost with respect to $W^{(2)}$|(hiddenLayerSize, outputLayerSize)|\n",
    "|dz3dz2|$$\\frac{\\partial z^{(3)}}{\\partial z^{(2)}}$$|Partial derivative of $z^{(3)}$ with respect to $z^{(2)}$|(numExamples, hiddenLayerSize)|\n",
    "|dJdW1|$$\\frac{\\partial J}{\\partial W^{(1)}}$$|Partial derivative of cost with respect to $W^{(1)}$|(inputLayerSize, hiddenLayerSize)|\n",
    "|delta2|$$\\delta^{(2)}$$|Backpropagating Error 2|(numExamples,hiddenLayerSize)|\n",
    "|delta3|$$\\delta^{(3)}$$|Backpropagating Error 1|(numExamples,outputLayerSize)|"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " we  use gradient descent to train our Neural Network, so it could make better predictions of your score on a test based on how many hours you slept, and how many hours you studied the night before. To perform gradient descent, we need an equation and some code for our gradient, dJ/dW.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our weights, W, are spread across two matrices, W1 and W2. We’ll separate our dJ/dW computation in the same way, by computing dJdW1 and dJdW2 independently. We should have just as many gradient values as weight values, so when we’re done, our matrices dJdW1 and dJdW2 will be the same size as W1 and W2."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\frac{\\partial J}{\\partial W^{(2)}} = \\frac{\\partial \\sum \\frac{1}{2}(y-\\hat{y})^2}{\\partial W^{(2)}}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let’s work on dJdW2 first. The sum in our cost function adds the error from each example to create our overall cost. We’ll take advantage of the sum rule in differentiation, which says that the derivative of the sums equals the sum of the derivatives. We can move our sigma outside and just worry about the derivative of the inside expression first. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\frac{\\partial J}{\\partial W^{(2)}} = \\sum \\frac{\\partial \\frac{1}{2}(y-\\hat{y})^2}{\\partial W^{(2)}}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To keep things simple, we’ll temporarily forget about our summation. Once we’ve computed dJdW for a single example, we’ll add all our individual derivative terms together. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now evaluate our derivative. The power rule tells us to bring down our exponent, 2, and multiply. To finish our derivative, we’ll need to apply the chain rule. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The chain rule tells us how to take the derivative of a function inside of a function, and generally says we take the derivative of the outside function and then multiply it by the derivative of the inside function. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One way to express the chain rule is as the product of derivatives, this will come in very handy as we progress through backpropagation. In fact, a better name for backpropagation might be: don’t stop doing the chain rule. ever. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We’ve taken the derivative of the outside of our cost function - now we need to multiply it by the derivative of the inside."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Y is just our test scores, which won’t change, so the derivative of y, a constant, with respect to W two is 0! yHat, on the other hand, does change with respect to W two, so we’ll apply the chain rule and multiply our results by minus dYhat/dW2. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\frac{\\partial J}{\\partial W^{(2)}} = -(y-\\hat{y}) \\frac{\\partial \\hat{y}}{\\partial W^{(2)}}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now need to think about the derivative of yHat with respect to W2. Equation 4 tells us that yHat is our activation function of z3, so we it will be helpful to apply the chain rule again to break dyHat/dW2 into dyHat/dz3 times dz3/dW2. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\frac{\\partial J}{\\partial W^{(2)}} = \n",
    "-(y-\\hat{y})\n",
    "\\frac{\\partial \\hat{y}}{\\partial z^{(3)}}  \n",
    "\\frac{\\partial z^{(3)}}{\\partial W^{(2)}}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To find the rate of change of yHat with respect to z3, we need to differentiate our sigmoid activation function with respect to z. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "f(z) = \\frac{1}{1+e^{-z}}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "f^\\prime(z) = \\frac{e^{-z}}{(1+e^{-z})^2}\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Whole Class with additions:\n",
    "class Neural_Network(object):\n",
    "    def __init__(self):        \n",
    "        #Define Hyperparameters\n",
    "        self.inputLayerSize = 2\n",
    "        self.outputLayerSize = 1\n",
    "        self.hiddenLayerSize = 3\n",
    "\n",
    "        #Weights (parameters)\n",
    "        self.W1 = np.random.randn(self.inputLayerSize,self.hiddenLayerSize)\n",
    "        self.W2 = np.random.randn(self.hiddenLayerSize,self.outputLayerSize)\n",
    "\n",
    "    def forward(self, X):\n",
    "        #Propogate inputs though network\n",
    "        self.z2 = np.dot(X, self.W1)\n",
    "        self.a2 = self.sigmoid(self.z2)\n",
    "        self.z3 = np.dot(self.a2, self.W2)\n",
    "        yHat = self.sigmoid(self.z3) \n",
    "        return yHat\n",
    "\n",
    "    def sigmoid(self, z):\n",
    "        #Apply sigmoid activation function to scalar, vector, or matrix\n",
    "        return 1/(1+np.exp(-z))\n",
    "\n",
    "    def sigmoidPrime(self,z):\n",
    "        #Gradient of sigmoid\n",
    "        return np.exp(-z)/((1+np.exp(-z))**2)\n",
    "\n",
    "    def costFunction(self, X, y):\n",
    "        #Compute cost for given X,y, use weights already stored in class.\n",
    "        self.yHat = self.forward(X)\n",
    "        J = 0.5*sum((y-self.yHat)**2)\n",
    "        return J\n",
    "\n",
    "    def costFunctionPrime(self, X, y):\n",
    "        #Compute derivative with respect to W and W2 for a given X and y:\n",
    "        self.yHat = self.forward(X)\n",
    "\n",
    "        delta3 = np.multiply(-(y-self.yHat), self.sigmoidPrime(self.z3))\n",
    "        dJdW2 = np.dot(self.a2.T, delta3)\n",
    "\n",
    "        delta2 = np.dot(delta3, self.W2.T)*self.sigmoidPrime(self.z2)\n",
    "        dJdW1 = np.dot(X.T, delta2)  \n",
    "\n",
    "        return dJdW1, dJdW2\n",
    "\n",
    "    #Helper Functions for interacting with other classes:\n",
    "    def getParams(self):\n",
    "        #Get W1 and W2 unrolled into vector:\n",
    "        params = np.concatenate((self.W1.ravel(), self.W2.ravel()))\n",
    "        return params\n",
    "\n",
    "    def setParams(self, params):\n",
    "        #Set W1 and W2 using single paramater vector.\n",
    "        W1_start = 0\n",
    "        W1_end = self.hiddenLayerSize * self.inputLayerSize\n",
    "        self.W1 = np.reshape(params[W1_start:W1_end], (self.inputLayerSize , self.hiddenLayerSize))\n",
    "        W2_end = W1_end + self.hiddenLayerSize*self.outputLayerSize\n",
    "        self.W2 = np.reshape(params[W1_end:W2_end], (self.hiddenLayerSize, self.outputLayerSize))\n",
    "\n",
    "    def computeGradients(self, X, y):\n",
    "        dJdW1, dJdW2 = self.costFunctionPrime(X, y)\n",
    "        return np.concatenate((dJdW1.ravel(), dJdW2.ravel()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def sigmoid(z):\n",
    "    #Apply sigmoid activation function to scalar, vector, or matrix\n",
    "    return 1/(1+np.exp(-z))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def sigmoidPrime(z):\n",
    "    #Derivative of sigmoid function\n",
    "    return np.exp(-z)/((1+np.exp(-z))**2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEACAYAAACj0I2EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XlcFfX6wPHPl03BDUFFERXcccU0c01s07KrddVcyrLF\n9KdWt67lkilo3ZZr92ppVmYulVpqN8vcSiW3NHMtMXdFhUxF3FLW7++PIURFOMKcM+cMz/v1Oi/O\nnPNlzvM48Dg8M/MdpbVGCCGE5/OyOgAhhBDmkIIuhBA2IQVdCCFsQgq6EELYhBR0IYSwCSnoQghh\nEwUWdKXUdKXUCaXUznzGvKOU2qeU2q6UijI3RCGEEI5wZA99BtDpRm8qpe4Fammt6wADgfdNik0I\nIcRNKLCga63XAWfyGdINmJ09dhNQTikVYk54QgghHGVGD70qcDTX8vHs14QQQriQGQVd5fGazCcg\nhBAu5mPCOo4B1XIthwGJeQ1USkmhF0KIQtBa57XzfBVH99AVee+JA3wNPAqglGoFpGitT+QTlG0f\nY8eOtTwGyU9yk/w0WVmakyc127ZpFi/WvP++5pVXNI8/rrnnHk1UlKZqVY2fn8ZoKNzco1Mn1+bn\nqAL30JVSc4BoIFgplQCMBfyM2qw/1FovUUrdp5TaD1wEHnf4023m8OHDVofgVHbOz865gT3zS02F\nQ4dg/35YsuQwyclw4IDxOHzYeN8RAQFQsaLxqFDhytfy5aFcOQgMNB65n5cv79TUCq3Agq617uvA\nmKHmhCOEEFdLT4d9++DXX69+HDgAWVlXxm3efPX3lSsHYWFQter1XytXvlK4AwJcm48zmdFDF9n6\n9+9vdQhOZef87JwbeE5+mZnw229Gcf7pJ+Prjh1GUb+WtzfUrAm1akHJkv25/Xbjea1aEBEBZcq4\nPn6rqZvpzxT5w5TSrvw8IYR7S001CndcnPH46Se4cOH6cRER0KjR1Y969aBECVdHbA2lFNqBg6JS\n0E0UFxdHdHS01WE4jZ3zc8fcwsPDOXLkiNVhCBeqUaNGnsc7HC3o0nIRwk0dOXLkps5wEJ5PqQJr\ndv7fL3voQrin7L0yq8MQLnSjbe7oHrpMnyuEEDYhBd1EcXFxVofgVHbOz865ieJDCroQwjSvv/46\nTz/9tNt9bkREBKtWrXJhRNaQHroQbkp66OaJiIhg+vTp3HHHHVaHki/poQshhACkoJvK7n1YO+dn\n59yc5c033yQsLIyyZcsSGRnJ6tWriY2NpV+/fjljZs+eTXh4OBUrVuTVV1+9qvURGxvLQw89RL9+\n/ShbtixNmzZl3759vPHGG4SEhFCjRg2+//77nHUlJSXRrVs3goODqVu3Lh999FHOe9d+7ieffJLz\nuf/6179c8K/hHqSgC+GBlDLvURh79+5lypQpbNmyhXPnzrF8+XLCw8OzYzNWGh8fz5AhQ5g7dy5J\nSUmcPXuWxMSrZ9ZevHgxjz32GCkpKURFRdGpUye01iQmJvLKK69c1Rfv3bs31atX5/fff2f+/PmM\nGjWK1atX5/o3ufK5gwcP5rPPPiMxMZHTp09z/PjxwiXqYaSgm8jdrjQ0m53zs3NuzuDt7U1aWhq/\n/vorGRkZVK9enYiIiKvGLFy4kK5du9K6dWt8fHwYN27cdetp3749d911F15eXvTs2ZNTp04xYsQI\nvL296d27N0eOHOHcuXMcPXqUDRs28Oabb+Lr60vTpk156qmn+OSTT65b58KFC/nb3/5G27Zt8fX1\nZfz48UW+YMdTSEEXwgNpbd6jMGrVqsXEiROJiYmhUqVK9O3bl6SkpKvGJCYmUq3alXvf+Pv7Exwc\nfNWYkJCQq96vUKFCTvH19/dHa82FCxdISkoiKCiIgFxTI9aoUSPPPe9rPzcgIOC6z7UrKegmsnsf\n1s752Tk3Z+nduzdr164lISEBgOHDh1/1fpUqVTh27FjO8qVLlzh9+nShPis0NJTk5GQuXryY81pC\nQgJVq15/++IqVapw9OiV2xz/+eefhf5cTyMFXQhx0/bu3cvq1atJS0vDz88Pf39/fHyunhqqR48e\nfPPNN2zcuJH09HTGjh1b6M8LCwujTZs2jBw5ktTUVHbu3Mn06dN55JFHrhvbo0cPFi9ezIYNG0hP\nT2fMmDHF5vRPKegmsnsf1s752Tk3Z0hNTWXEiBFUrFiR0NBQTp48ed3ZJA0aNODdd9+lV69ehIaG\nUq5cOSpVqkSJm5jzNnfve+7cuRw6dIjQ0FC6d+/O+PHj8zyvvEGDBkyZMoU+ffoQGhpKcHAwYWFh\nhU/Wg8iFRUK4KbtdWHTx4kUCAwPZv38/NWrUsDoctyQXFrkRu/dh7ZyfnXOz0uLFi7l06RIXL17k\nn//8J02aNJFi7kRS0IUQTrNo0SJCQ0MJCwvjwIEDzJs3z+qQbE1aLkK4Kbu1XETBpOUihBACkIJu\nKrv3Ye2cn51zE8WHFHQhhLAJ6aEL4aakh178SA9dCCEEIAXdVHbvw9o5Pzvn5kp2vAXdnDlz6Ny5\nc2FDcykp6EII04wcOZIPP/zQrT83NjYWPz8/ypYtS1BQEO3atWPjxo03HN+3b1+WLVtmVqhOJQXd\nRHafD8TO+dk5N3G93r17c+7cOU6ePEnbtm35+9//nue4zMxMF0dWNFLQhRCFYodb0Hl7e/PYY49x\n4sQJkpOTmTVrFu3ateOFF14gODiY2NhYZs2aRfv27XO+x8vLi6lTp1K3bl3KlSvHmDFjOHjwIG3a\ntCEwMJDevXuTkZGRM37x4sU0a9aM8uXL065dO3755Zei/cPnw6fgIcJRcXFxtt7Ts3N+npabijXv\nDjx67M2fSZP7FnQhISEkJCSQmZnJmjVrrrsF3YoVK7j11lsZOXJknreg+/rrr5k1axaPP/44nTp1\nYsCAASQmJjJjxgyefvppDh48CBh71U2aNOH3338nPj6eu+++m1q1atGxY0fg+lvQLVu2jJYtWzJi\nxIgb3oIuNTWVGTNmEBYWRlBQEACbNm2ib9++nDx5kvT0dObNm3fdHY+WL1/Otm3bSEhIoFmzZvz4\n44/MnTuXoKAgWrVqxdy5c+nXrx9bt27lySef5Ntvv6V58+Z8+umndO3alb179+Lr63vT/+4FkT10\nIcRN8/Rb0H3++ecEBQVRo0YNtm3bxqJFi3Leq1q1KoMHD8bLy+uGU/2OGDGCUqVKERkZSaNGjbjn\nnnuoUaMGZcqU4d5772Xbtm0AfPTRRwwaNIgWLVqglKJfv36UKFEi3559Ucgeuok8aQ+vMOycn6fl\nVpi9ajPlvgXdrl276Ny5M2+//fZVY1xxC7otW7ZcF5sjt6Dr1asXs2fPzjO33N97I5UqVboq7mvz\nOHHiBABHjhxh9uzZvPvuuwBorUlPT7/uLxWzyB66EKJQ7HoLOjNvKF2tWjVefvllkpOTSU5O5syZ\nM1y4cIFevXqZ9hm5SUE3kd3PZbZzfnbOzRnkFnSOGTBgAO+//z4//fQTYNzkY8mSJVf9x2QmKehC\niJtWnG9Bd+0efH579M2bN2fatGkMHTqUoKAg6taty6xZs0yL5brYHPmfSynVGZiI8R/AdK31m9e8\nXw2YBQRmjxmptV6ax3pkLhchHGS3uVzkFnQFc/pcLkopL2Ay0AloCPRRStW/Ztho4HOt9S1AH+A9\nB2IXQtic3ILOtRxpubQE9mmtj2it04F5QLdrxmQBZbOfBwJ5n/Rpc3bvw9o5PzvnZiW5BZ1rOXLa\nYlXgaK7lYxhFPrdYYIVS6lkgALjLnPCEEJ5s2rRpTJs2zeowig1HCnpefZtrmzx9gBla6/8qpVoB\nn2K0Z67Tv39/wsPDAQgMDCQqKirnHOC/9pI8dfmv19wlHsnP8eXo6Gi3ikcUb3FxccycORMgp146\nosCDotkFOkZr3Tl7eQSgcx8YVUr9CnTSWh/PXj4A3Ka1PnXNuuSgqBAOOHQIata010FRUTBX3OBi\nM1BbKVVDKeUH9Aa+vmbMEbLbLEqpSKDEtcW8OLB7H9bO+blTbmlp0Lu31VEIT1Rgy0VrnamUGgqs\n4Mppi7uVUrHAZq31YmAYME0p9TzGAdLHnBm0EHY2ejT89BMEBNQw9apF4f6KegaQ3FNUCDeydCnc\ndx94e8OaNdCmjdURCXcg9xQVwsMkJcFj2X/bjh8vxVzcPCnoJnKnPqwz2Dk/q3PTGp54Ak6ehLvu\ngmvmuSoyq/NzNrvn5ygp6EK4gQ8/hGXLICgIZs8GL/nNFIUgPXQhLHbgADRtChcvwrx54KSZVYUH\nkx66EB4gMxP69zeKea9eUsxF0UhBN5Hd+3h2zs+q3P7zH1i3DqpUgSlTnPc5dt52YP/8HCUFXQiL\n7NplnHMO8NFHcM1d0oS4adJDF8ICmZnQrh1s3AhPPQUyf5XIj/TQhXBjU6caxTw0FCZMsDoaYRdS\n0E1k9z6enfNzZW5Hj8LIkcbzyZOhXDnnf6adtx3YPz9HSUEXwoW0hsGD4cIFePBB4yGEWaSHLoQL\nffGFcWpi2bKwe7fRchGiINJDF8LNnDkDzz5rPH/rLSnmwnxS0E1k9z6enfNzRW4jRsCJE9C+PQwY\n4PSPu4qdtx3YPz9HSUEXwgV+/tk4NdHHB95/X+ZqEc4hPXQhnCwrC1q3Nm5aMWwY/PvfVkckPI2j\nPXQp6EI42ccfw5NPGj3z336DMmWsjkh4GjkoagG79/HsnJ+zcjtz5src5hMmWFfM7bztwP75OUoK\nuhBONGYMnDoFHTrIjZ+F80nLRQgn2b4dmjcHpWDbNmjc2OqIhKeSlosQFtIannnGOCA6dKgUc+Ea\nUtBNZPc+np3zMzu3+fONec4rVYLYWFNXXSh23nZg//wcJQVdCJNdvnzlQOirr7pm8i0hQHroQpju\nrbeMgt6okdFH9/a2OiLh6eQ8dCEscPIk1K4N587B8uVwzz1WRyTsQA6KWsDufTw752dWbjExRjG/\n9173KuZ23nZg//wcJQVdCJPEx8MHHxjztMjl/cIK0nIRwiRdusCSJTBokHGLOSHMIj10IVzou++M\nFkuZMrB/v3G6ohBmkR66Bezex7NzfkXJLSsLXnzReD5qlHsWcztvO7B/fo6Sgi5EEc2bBzt2QFgY\nPPec1dGI4kxaLkIUQVoaREbCwYMwfTo88YTVEQk7kpaLEC4wbZpRzOvXh0cftToaUdxJQTeR3ft4\nds6vMLlduADjxxvPX3vNuL2cu7LztgP75+coKehCFNKkScZNn1u2hAcftDoaIRzsoSulOgMTMf4D\nmK61fjOPMQ8BY4EsYIfW+pE8xkgPXdjC6dNQs6ZxVeiqVdCxo9URCTtztIde4B+JSikvYDJwJ5AI\nbFZKLdJa/5ZrTG1gONBaa31OKVWh8KEL4f5ef90o5vfcI8VcuA9HWi4tgX1a6yNa63RgHtDtmjED\ngCla63MAWutT5obpGezex7NzfjeT29GjMHmy8fz1150Tj9nsvO3A/vk5ypGCXhU4mmv5WPZrudUF\n6iml1imlNiilOpkVoBDuJiYGUlOhVy+45RaroxHiigJ76EqpHsA9Wuuns5cfAW7VWj+Xa8w3QBrQ\nE6gOrAUa/rXHnmuc9NCFR9u925jnXCnjeZ06VkckigPTeugYe+TVcy2HYfTSrx3zo9Y6CzislNoD\n1AG2XLuy/v37Ex4eDkBgYCBRUVFER0cDV/5skmVZdtflceMgKyuap5+G48fjOH7cveKTZXssx8XF\nMXPmTICceukQrXW+D8Ab2A/UAPyA7UDkNWM6ATOzn1cAjgDl81iXtrPVq1dbHYJT2Tk/R3L75Ret\nldLaz0/rhATnx2QmO287re2fX3btLLBeF9hD11pnAkOBFcAuYJ7WerdSKlYpdX/2mOXAaaXULmAl\nMExrfcbx/1aEcH+xsaA1DBgA1apZHY0Q15O5XIRwwM6d0LQplCgBBw5A1WtPCxDCiWQuFyFMFBtr\nfB04UIq5cF9S0E3010ENu7Jzfvnltn07fPkllCwJI0a4LiYz2Xnbgf3zc5QUdCEKEBNjfB00CKpU\nsTQUIfIlPXQh8rFlC7RoAf7+xjS5lStbHZEojqSHLoQJ/to7HzxYirlwf1LQTWT3Pp6d88srt82b\nYfFiCAiAl15yfUxmsvO2A/vn5ygp6ELcwF9750OHuueNn4W4lvTQhcjDxo3QujWUKgWHD0MFmRBa\nWEh66EIUwV97588+K8VceA4p6Cayex/Pzvnlzm3DBli+HMqUgX/+07qYzGTnbQf2z89RUtCFuMbY\nscbXZ5+F4GBrYxHiZkgPXYhc1q6F22+HsmXh0CEICrI6IiGkhy5Eofy1d/6Pf0gxF55HCrqJ7N7H\ns3N+cXFx/PADrF4N5crB889bHZG57LztwP75OUoKuhAY85yPGWM8f+EFCAy0Nh4hCkN66EIAq1bB\nnXcahfzwYWMvXQh3IT10IRyk9ZXe+bBhUsyF55KCbiK79/Hsmt/KlbBuXRxBQfDMM1ZH4xx23XZ/\nsXt+jpKCLoq13L3zYcOM0xWF8FTSQxfF2vLl0LmzcXn/wYPG1aFCuBvpoQtRgNy98xdflGIuPJ8U\ndBPZvY9nt/yWLoVNm6BiRWjcOM7qcJzKbtvuWnbPz1FS0EWxlHvvfPhw4xZzQng66aGLYumbb6Br\nVwgJMXrnAQFWRyTEjUkPXYgbuHbvXIq5sAsp6Cayex/PLvktWgTbtkGVKjBokPGaXXK7EcmveJCC\nLoqVrKwre+ejRknvXNiL9NBFsbJgAfTsCWFhsG8flCxpdURCFEx66EJcIzPzyt75yy9LMRf2IwXd\nRHbv43l6fvPnQ3w8VK8OTzxx9XuenltBJL/iQQq6KBYyMyEmxng+ejT4+VkajhBOIT10USx8+in0\n6wcREbBnD/j6Wh2REI6THroQ2TIyYNw44/krr0gxF/YlBd1Edu/jeWp+n31mnNFSq5axl54XT83N\nUZJf8SAFXdhaevqVvfOxY8HHx9p4hHAm6aELW5s+HZ56CurVg19/lYIuPJOpPXSlVGel1G9Kqb1K\nqeH5jOuhlMpSSt1yM8EK4QxpaTB+vPFc9s5FcVBgQVdKeQGTgU5AQ6CPUqp+HuNKA88AG80O0lPY\nvY/nafl9/DEcOQKRkfDQQ/mP9bTcbpbkVzw4sofeEtintT6itU4H5gHd8hg3HngTSDUxPiEK5dKl\nK3vnMTHg7W1pOEK4RIE9dKVUd6CT1vrp7OVHgJZa62dzjYkCXtZa91RKrQb+qbXemse6pIcuXGLC\nBOO2cs2awc8/g5cc/hcezNEeuiNdxbxWklOVlVIK+C/wWAHfI4RLnD0Lr79uPH/tNSnmovhwpKAf\nA6rnWg4DEnMtl8HorcdlF/fKwCKlVNe89tL79+9PeHg4AIGBgURFRREdHQ1c6YN56vLEiRNtlY+n\n5rdqVTTJycZ9Qo0JuAr+/tw9WKvjd8ay5OdZy3FxccycORMgp146RGud7wPwBvYDNQA/YDsQmc/4\n1UCzG7yn7Wz16tVWh+BUnpDfiRNalyqlNWi9bp3j3+cJuRWF5OfZsmtngfXaofPQlVKdgUkYB1Gn\na63fUErFApu11ouvGbsKGKalhy4s8I9/wKRJ0KULLF5c8HghPIGjPXS5sEjYRkIC1KljnH++fTs0\nbWp1REKYQybnskDuPp4duXt+sbFGMe/d++aLubvnVlSSX/EgBV3Ywm+/wcyZxvnmf83dIkRxIy0X\nYQs9exr3C336afjgA6ujEcJc0kMXxcaPP0KbNsY9QvftM24ALYSdSA/dAnbv47ljflrDsGHG8xde\nKHwxd8fczCT5FQ9S0IVH+9//YMMGqFgRht9wHlAhigdpuQiPlZYGDRvC/v0weTIMGWJ1REI4h7Rc\nhO198IFRzOvWNQ6GClHcSUE3kd37eO6U39mzxnnnAG++WfQbP7tTbs4g+RUPUtCFR3r9dTh9Gtq3\nh255zc4vRDEkPXThcRISjDZLaips2gQtW1odkRDOJT10YVsjRxrFvHdvKeZC5CYF3UR27+O5Q37r\n18OcOVCiBPzrX+at1x1ycybJr3iQgi48RmYmPPOM8fyllyAiwtp4hHA30kMXHmPaNOP0xLAwYzKu\nUqWsjkgI15C5XIStpKQYc52fOgVz5xr9cyGKCzkoagG79/GszC8mxijm7dtDr17mr1+2nWeze36O\nkoIu3F58vHFpv5cXvPMOqAL3U4QonqTlItya1nD33bByJQwaBFOnWh2REK4nPXRhC3PmwMMPQ1AQ\n7NkDFSpYHZEQric9dAvYvY/n6vzOnIHnnzeev/WWc4u5bDvPZvf8HCUFXbitl1+GP/6Adu3g8cet\njkYI9yctF+GWNm2C1q2Nmz5v327Mey5EcSUtF+GxMjJg4MArt5eTYi6EY6Sgm8jufTxX5ffOO7Bj\nB4SHwyuvuOQjZdt5OLvn5ygp6MKt7N8Po0cbzydPhoAAa+MRwpNID124jaws6NgR1qwxTlX89FOr\nIxLCPUgPXXicqVONYh4SApMmWR2NEJ5H9tBNFBcXR3R0tNVhOI0z8zt0CBo3hosXYeFC+PvfzV2/\n1po9p/ew+fhmtiRtYe/pvRw9d5TE84lczrjM5f2X8a3pS2DJQIL8g6hZvib1K9SnSUgT2ldvT3hg\nOMqD5xyQn03P5ugeuo8rghEiP1rDgAFGMX/oIfOKeVpmGsv3L+ebvd+wZN8Sjp8/fuPBWZCamcqJ\niyc4cfEEu0/t5tt93+a8HVY2jC51utCjQQ+iw6Px8ZJfHeF+ZA9dWO6DD4x5WipUMCbiqlixaOv7\n9Y9f+WjrR3z2y2ec+vNUzushpUJoU60NLUJb0LBiQ6qXq07VslUp5VsKP28/0rPSSbmcwsmLJ9mX\nvI89p/awOXEzaxPWknwpOWc9wf7BPB71OINaDKJWUK2iBSuEA2QuF+ER9u6FZs3gzz/h88+NPfTC\n0FoTdziON9e/yfIDy3Neb1SpEQ81eIgudbsQVTkKL3Xzh42ydBbbf9/Ol7u/ZEH8Avac3pPz3r21\n7+Xl9i/TtnrbwgUuhAMcLehorV32MD7OvlavXm11CE5ldn5paVq3aKE1aP3ww4Vfz+pDq3Wrj1pp\nYtDEoANeC9CDvhmkNx/frLOyshxbh4O5ZWVl6U3HNun+X/XXJV8tmfOZd866U685vKbwSTiZ/Gx6\ntuzaWWCNlUagsExMDPz8M9SoAVOm3Pz37/pjF8O/H57T6w72D+bZ255lyK1DCA4INjfYbEopWlZt\nScuqLZlw9wQmbZrEpE2TWHloJSsPreSB+g/w77v/Te2g2k75fCHyIy0XYYk1ayA62rhZxQ8/GBNw\nOepc6jlGrxrNlM1TyNJZlPYrzfC2w3m+1fOU8nP9jUZTLqcwceNEJmyYwMX0i/h6+fKPVv9g9O2j\nKVuirMvjEfYjPXThtlJSoGlTSEgwZlR89VXHvk9rzZe7v+TZZc+SeD4Rb+XNwOYDGRs9lkqlKjk3\naAcknU9i1KpRzNw+EzDOjJnaZSr3173f2sCExzP1wiKlVGel1G9Kqb1KqeF5vP+8UmqXUmq7Uuo7\npVS1wgTt6ew+n4QZ+WkNjz1mFPNbb4WxYx37vuPnjtN1Xld6zO9B4vlEbqt6G1ue3sKULlNMKeZm\n5FalTBVmdJvB5gGbuTX0Vo6dO8bf5v6N3gt688fFP4q8/qKQn83iocCCrpTyAiYDnYCGQB+lVP1r\nhm0Fmmuto4CFwL/NDlTYw4QJ8PXXEBhonNXi61vw98z7dR6NpjZi8d7FlC1Rlvfue4/1T6ynaeWm\nzg+4EFqEtuDHJ3/kP/f8hwDfAD7f9TmRUyJZGL/Q6tCEzRXYclFKtQLGaq3vzV4egXHE9c0bjI8C\n3tVat8/jPWm5FGNr1xpztWRmwqJF0LVr/uPPXDrD4CWDmffrPADuq3Mf0/42jdAyoS6I1hyHzhxi\n4OKBfHfwOwD6R/Xnnc7vUKZEGYsjE57EzJZLVeBoruVj2a/dyJPAUgfWK4qREyegVy+jmL/0UsHF\n/PuD39N4amPm/TqPAN8APrj/Axb3WexRxRwgonwEyx9ZzuR7J1PSpyQzt88k6oMoNhzdYHVowoYc\nOW0xr/8V8tzNVko9AjQHOtxoZf379yc8PByAwMBAoqKicuZg+KsP5qnLEydOtFU+ZuXXtm00ffpA\nUlIcjRvDa6/deHxmViYr9UpeX/c6HILIipF8PfJragfVdmp+uXuwzlj/kJZDKJVYilfXvMoBDtB+\nRnv6le1Hvyb9uPOOO03/PFfnZ/Wy3fKLi4tj5syZADn10iEFnagOtAKW5VoeAQzPY9xdwC4gOJ91\nmX7CvTux+8UNhc1vyBDj4qGQEK2PH7/xuOPnjuvbZ9yuiUF7xXrpcXHjdHpmeuGCvUmu2napGal6\n+HfDtYpRORck/X7+d6d/rvxsejYcvLDIkR66N7AHuBNIAn4C+mitd+ca0wyYD3TSWh/IZ126oM8T\n9vLXPC1+fhAXZ9wnNC/fHfiOh798mJN/nqRK6SrM7T6XDuE3/EPP4+XOt3Lpysz5+xw6RnS0Oizh\npkw9D10p1RmYhNFzn661fkMpFQts1lovVkp9BzTCKPgKOKK1fiCP9UhBL0bi4uDuu417hM6aBY8+\nev2YzKxMYn+I5dU1r6LR3FXzLj598FNCSoe4PF5XSzyfSN+FffnhyA94KS9iOsQwqv0ovL28rQ5N\nuBmZy8UCdv+z72byO3BA6+Bgo9UybFjeY5LOJ+mOMztqYtAqRunYuFidkZlhTrA3yaptl56Zrl9Z\n9UpOC+bu2XfrExdOmP458rPp2XCw5SJ3LBKmO3kSOneG06fhvvvgjTeuH7Pq0Cqi3o9i9eHVhJQK\n4ftHv2dMhzHFbu/Ux8uHcR3HseyRZVQMqMh3B78j6v0o1hxZY3VowgPJpf/CVBcvGueab94MUVHG\nPC1lc01nkqWzeG3Na8T8EEOWziI6PJq53edSuXRl64J2E4nnE+mzsA9rjqzBS3nx2h2v8VLblwo1\n5a+wF5nLRbhcejo88AAsWQLh4bBhA1SpcuX9kxdP8sj/HmHFgRUoFKNvH83YDmOL3V55fjKyMhiz\neoxx2iZMDt4FAAARg0lEQVTGxVSzH5jttNkjhWeQm0RbIPe5sHaUX35ZWfD000YxDw6G5cuvLuYb\njm6g2QfNWHFgBRUCKrD04aWM6zjObYq5u2w7Hy8f/nXnv/i277cE+QexZN8Smn3QjB+P/lik9bpL\nfs5i9/wcJQVdFJnW8MwzMHMmBATAt99C3bp/vaf5z4//ocPMDhw/f5w21dqwbeA2OtXuZGnM7u6+\nOvexfeB2Woe15ui5o9w+83b+8+N/kL9wRX6k5SKKRGt4/nmYNMk41/zrr6FTdq1OuZzCE4ue4H+/\n/Q+AF1q9wBt3vYGvtwMzcgkA0jPTGblyJG//+DYA3ep1Y0a3GZT3L29xZMKVpIcunE5rY16WCROM\nWRO/+so4qwVgW9I2eszvwcEzBylboiwzu83kwcgHrQ3Yg33121f0/6o/Z1PPEhEYwRc9v6BFaAur\nwxIuIj10C9i9j5c7P61hxAijmPv4wIIFRjHXWvPhlg9pPb01B88cpFnlZmx9eqvbF3N333YP1H+A\nrQO30rxKcw6lHKLtx22Z8tMUh1sw7p5fUdk9P0dJQRc3LTPTuJz/rbfA29uY17xrV2O6257zezJw\n8UBSM1MZ2HwgG57cQK2gWlaHbAs1y9dk/RPrGXrrUNIy0xi6dCi9F/bmXOo5q0MTbkJaLuKmpKVB\nv37wxRdQsiTMnw/33w9rj6zl4S8f5ui5o5TxK8P7979P38Z9rQ7Xtr7Y9QVPff0U59POUyeoDp/3\n+JxmVZpZHZZwEumhC9NdvAjduxunJJYtC998A23aZfDqmlcZv2Y8WTqL26rexpzuc6hZvqbV4dre\n3tN76Tm/JztP7MTXy5fxHcczrM0wtzkVVJhHeugWsHMf79gxaNo0juXLoWJFWL0aqjc5TPTMaGJ/\niEVrzah2o1j7+FqPLOaeuO3qBtdl45MbGXLrENKz0hmxcgQdZ3XkcMrh68Z6Yn43w+75OUoKuijQ\nzz9Dy5Zw4ADUrg1r12o2Z31A46mNWX90PaFlQln56Epeu/M1OSXRxfx9/Zl832SWPryUyqUrszZh\nLU2mNmH2jtlyznoxJC0Xka8FC4xpby9dguhomDTjKC+ue4oVB1YA0LNBT97r8h4VAipYG6jg1J+n\nGLh4IF/u/hKAHg16MOW+KVQqVcniyERRSQ9dFEl6OowcCW8b17PwxJOa1oNm8c/vn+Nc6jmC/YN5\nr8t7PNTwIWsDFVfRWjNrxyyeWfoMF9IuEOQfxMROE3mkySMoVfB02sI9SQ/dAnbp4x07ZsyY+Pbb\nxjnmL084wLEOnRkw+XHOpZ6jW71u7Bq8y1bF3C7bTilF/6j+7By0k7tq3kXypWQe/epRbht9G0dS\njlgdntPYZfsVlRR0cZWlS6FZM1i/HkKrpfHEjNd4+3IjVhxcQZkSZZj9wGz+1+t/xeKOQp4sonwE\nKx5ZwYxuMwgsGcjm45tp+F5D3t30LplZmVaHJ5xEWi4CgAsXYNgw4x6gAC16rOFc+0HsPWPcOvbR\npo8y4e4JVCxV0cIoRWH8fuF3nln6DAviFwAQVTmKd+99l3bV21kcmXCU9NCFw9avNw58HjwIvhWO\nEPnccHZmfg4Yp8ZN7TKVOyLusDhKUVRf/fYVzy17joSzCQA83Phh3rr7LULLhFocmSiI9NAt4Gl9\nvJQUGDIE2reHg8fOU6nPKNSz9diZ+TklfUoytsNYdgzakVPMPS2/m2Hn3MDI74H6D7B7yG7G3D6G\nEt4l+OyXz6g3uR5vrHuDS+mXrA6xSOy+/RwlBb0Y0hrmzYPISHjv/XRUiw8pNbIOf9R7nbSsVPo2\n7sueoXuIiY6hpE9Jq8MVJgrwDSC2Yyy7h+zmgfoPcCHtAiNXjqT2u7WZtmUaGVkZVocoikBaLsXM\ntm3w4ouwcnUGNP6MEp3GkRpwEIBWYa34b6f/0iqslcVRClf57sB3DP9+ONt+3wYYLbbX7niN7pHd\n5TRHNyI9dHGVo0dh9GiY/WkmNJiP1x0xZAXtAYxf4nHR43io4UPyS1wMZeks5u+az+jVo9mfvB+A\npiFNGdV+FN0ju8vcMG5AeugWcMc+3u+/G3vkdRpcYnb8VBhaH3r0IStoDxGBEczsNpNdg3fRq1Gv\nAou5O+ZnFjvnBvnn56W86NWoF/GD45naZSpVSldhx4kd9FrQi8gpkXy87WPSMtNcF2wh2H37OUoK\nuk0dPw7PPQfhDU4xYfM4Uv+vOtw/GIL2Ex4Yzof3f8ieoXt4LOoxfLx8rA5XuAFfb18GtRjEwecO\n8n6X94kIjGBf8j6e/PpJar9Tm7fWv8XpP09bHabIh7RcbGbnTpj0jmb2qo1kNP0AGn4OvpcBaBHa\nghfbvMjfI/8uRVwUKCMrg3m/zuP1da8TfzIegJI+JenbqC9DWw6V+dddSHroxUhGhjE3+duTz7H+\n3GfQ4n2ovDPn/Xtr38tLbV+iQ40O0iMXNy1LZ7Fs/zIm/zSZpfuX5rzeplobHo96nJ4NelKuZDkL\nI7Q/KegWiIuLIzo62mWft2cPfDwrjY/ilpFc9TOo93XO3nhQiYoMaPEEA24ZYNot4FydnyvZOTcw\nL799p/fx3ub3+Hj7xzm3vvP38efByAfp37Q/d0TcYclBVLtvP0cLuvzd7WFOnoT5C9OZsngN8Wo+\nNJgPnZJz3m8X1oEhtw3iwfoPUsKnhIWRCjuqE1yH/3b+L+PvGM+C+AXM2jGLuMNxzPllDnN+mUPl\n0pV5sP6DdI/sTofwDtLaczHZQ/cAx4/D3C/PM2PtMuIzF0Gdb8E/Jef9mqUbM+C2h+nbuA/Vy1W3\nMFJRHB1OOczsHbOZvWM2B84cyHk92D+YbvW68UD9B+gY0ZHSfqUtjNKzScvFg6Wnw4aNmXzy3VaW\n7/+eY74rofpa8Lly6liobyR9b3mAfs160ySkiYXRCmHQWrPt920sjF/Iwt0L2XN6T857vl6+tKve\njs61O9OpVieahDSR4zk3QQq6BQrbx8vMhK070pm/ZifLfv2R3ZdWk1Ft1VV74WhFXf+2PNKiG72j\nulEnuI55gTvIzn1KO+cGrs9Pa038yXgW7l7Ikn1L+On4T2iu/O6HlAqhfY32tKvWjvY12tMkpEmR\n2jN2337SQ3djZ85olm9IYtHPP7Px2I8c1RvIrLwZfC9BtSvjymbWpE3IXfRtfRf31u8ot3kTHkMp\nRcNKDWlYqSFjOowh+VIy3x/8nmX7l7H8wHISzyeyIH5BzpS+pf1K06ZaG1pVbcUtVW6heWhzqpap\nKnvxN0n20J1IazhwOI1vN/3GD3t28MsfOziWsYPL5XZAqZPXjS+TVodG5VrTtVk7erW4i4jyERZE\nLYRzaa3Zc3oP6xLWsS5hHWsT1nLwzMHrxlUMqEjz0ObcUvkWGoc0JrJCJPUq1CuWE8ZJy8WFzl/I\nYv3ORNbt3sv2Y3vYf2YvSWl7Oe+3F13uEHhdf4cY7/RAqtCMW6u0ptstrenStJXsgYtiK/F8IusT\n1vNz4s9sSdrC1qStnLl85rpxXsqLmuVrElkhksgKkdSvUJ+I8hFEBEYQVjbMtvPOmFrQlVKdgYkY\nUwVM11q/ec37fsBsoDlwCuiltU7IYz0eV9DT0zUHjp/ll0NJ7DySwN4TRzmSkkDSnwmcyUrgom8C\nWaWOgU8qHAKu3anWCv/LNQnzjaJppaZ0iGzKfc2iiAiq5nF/Ttq5T2nn3MDz8tNaczjlMFuTtrI1\naSvxp+KJPxnPgeQDZOo8bqF3CHxq+VC9XHUiAiOoWb4mNcrVILRM6FWPIP8gj/u9AxN76EopL2Ay\ncCeQCGxWSi3SWv+Wa9iTQLLWuo5SqhfwFtC7cKE7T1aW5lTKZY78kcKxU2dITE7h97MpnDibzPGU\nPzhx8QTJqX9wLvMEF/mDNN8TZPn/cdXZJQCUzH7k4n25IiWOlKJWtTuoV6EuzWvUo32DujSvWdM2\nfyJu377do4rCzbBzbuB5+SmljD3v8hF0b9A95/XUjFT2Je9j98ndxJ+MZ1/yPg6lHGLnlp1ciLjA\nwTMHOXjmICsPrcxzvX7efjnFvVKpSlTwr0BwQDAVAioQ7J/9NddyYMlAj9rrd+SgaEtgn9b6CIBS\nah7QDchd0LsBY7OfL8D4D8Ayf5y5SPNxT/JnVgqpKoU0rzNk+Kag/VKuL8655VGoAVRaGXzTQiib\nVZ2KftWpWqY6tSpUp0FodZrVqk6zmtUoXSKAmJgYYmJinJWW5VJSUgoe5KHsnBvYJ78SPiVoVKkR\njSo1uur1mIQYho8azuGUwxxKOcShM4c4eu4oiecTr3qcTT3L4ZTDHE457PBnlvYrTbkS5ShXslzO\n1zZhbXilwysmZ1d0jhT0qsDRXMvHMIp8nmO01plKqRSlVJDWOhkLlPL35Vjg53m/mVECr7Ty+GYE\n4qcDKUkgpbzLE1wihEqlKhFarhI1gkOoGVKJemEh1K1akbL+Aa5NQAhx0/x9/YmsGElkxcgbjvkz\n/U+Szidx/PxxTl48yelLpzn15ylO/3maU5eyv/55Kuf1s5fPciHtAhfSLnD8/PGc9ZTwds+rsB0p\n6Hn1ba5thF87RuUxxmVKlfTjH2HzqFC6LJXLB1I1qDxhFQOpUSmQMv7Oa38cPnzYaet2B3bOz865\ngeT3lwDfAGoF1XJ4fqMsncWFtAucvXyWs6lnSbmcwtnLZwnyDypCtM5T4EFRpVQrIEZr3Tl7eQSg\ncx8YVUotzR6zSSnlDSRprSvlsS7POiIqhBBuwqwLizYDtZVSNYAkjIOdfa4Z8w3wGLAJ6AmsKmxA\nQgghCqfAgp7dEx8KrODKaYu7lVKxwGat9WJgOvCJUmofcBo3PMNFCCHszqUXFgkhhHAeS+4pqpR6\nRin1m1LqF6XUG1bE4GxKqWFKqSyllHsePSkEpdRbSqndSqntSqmFSqmyVsdkBqVU5+yfx71KqeFW\nx2MmpVSYUmqVUio++/ftWatjMptSyksptVUp9bXVsZhNKVVOKTU/+/dul1LqtvzGu7ygK6Wigb8B\njbTWjYEJro7B2ZRSYcBdwBGrYzHZCqCh1joK2AeMtDieIst14VwnoCHQRylV39qoTJUBvKC1bgC0\nBobYLD+A54B4q4NwkknAEq11JNAU2J3fYCv20P8PeENrnQGgtT5lQQzO9l/gRauDMJvW+nutdVb2\n4kYgzMp4TJJz4ZzWOh3468I5W9Ba/6613p79/AJGQahqbVTmyd55ug/4yOpYzKaUKgO011rPANBa\nZ2itz+X3PVYU9LrA7UqpjUqp1UqpFhbE4DRKqb8BR7XWv1gdi5M9ASwtcJT7y+vCOdsUvNyUUuFA\nFMbZaHbx186THQ8G1gROKaVmZLeUPlRK+ef3DU6ZD10p9R0QkvsljH/w0dmfGai1bqWUuhX4Ijtw\nj1FAfqOAu695z2Pkk9vLWutvsse8DKRrredYEKLZHLlwzuMppUpjTMvxXPaeusdTSnUBTmitt2e3\ncj3qd80BPsAtwBCt9c9KqYnACK5Ms5LnN5hOa333jd5TSg0Cvswetzn7wGGw1vq0M2Jxhhvlp5Rq\nBIQDO5QxpVsYsEUp1VJr/YcLQyy0/LYdgFLqMYw/ce9wTUROdwzIfSPWMIxJ6GxDKeWDUcw/0Vov\nsjoeE7UFuiql7gP8gTJKqdla60ctjsssxzD+2v85e3kBkO9BeytaLl9hzNyIUqou4OtJxTw/Wutf\ntdaVtdY1tdYRGBukmacU84JkT6P8EtBVa51qdTwmyblwLnsa6N6A3c6W+BiI11pPsjoQM2mtR2mt\nq2uta2Jst1U2KuZorU8AR7PrJBh1M9+Dv1bcgm4G8LFS6hcgFbDNBsiDxl5/Br4L+AHfZc8pvVFr\nPdjakIrmRhfOWRyWaZRSbYGHgV+UUtswfiZHaa2XWRuZcNCzwGdKKV/gIPB4foPlwiIhhLAJSy4s\nEkIIYT4p6EIIYRNS0IUQwiakoAshhE1IQRdCCJuQgi6EEDYhBV0IIWxCCroQQtjE/wPzyakvHkAy\nXgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fea5d51ec10>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "testValues = np.arange(-5,5,0.01)\n",
    "plt.plot(testValues, sigmoid(testValues), linewidth=2)\n",
    "plt.plot(testValues, sigmoidPrime(testValues), linewidth=2)\n",
    "plt.grid(1)\n",
    "plt.legend(['sigmoid', 'sigmoidPrime'])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def tanh(x):\n",
    "    return np.tanh(x)\n",
    "\n",
    "def tanh_deriv(x):\n",
    "    return 1.0 - np.tanh(x)**2\n",
    "\n",
    "def logistic(x):\n",
    "    return 1/(1 + np.exp(-x))\n",
    "\n",
    "def logistic_derivative(x):\n",
    "    return logistic(x)*(1-logistic(x))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEACAYAAACwB81wAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd8VGX2+PHPk0boKXQIAelNkI4IRlDM2igKAipGfrAr\niLoq2Auufldd1xV0VwUEKavSLKj0BSIISg9KFYGEQOgdAiSZeX5/3CSEkDaZO3Nn7pz36zWvaXfu\nPSczObk589znKq01Qggh7C/I6gCEEEJ4hxR8IYQIEFLwhRAiQEjBF0KIACEFXwghAoQUfCGECBCm\nFHyl1GSl1BGl1K9FLPOBUmq3UipJKdXGjO0KIYQoObP28D8Dbi/sSaXUn4AGWutGwF+AT0zarhBC\niBIypeBrrX8CThWxSG9gevaya4HKSqnqZmxbCCFEyXirh18bSM1z/2D2Y0IIIbzEWwVfFfCYzOkg\nhBBeFOKl7RwAYvLcrwOk5V9IKSV/BIQQohS01gXtWF/FzIKvKHhPHuA74DFgllKqM3Baa32koAXt\nPJnb2LFjGTt2rNVheIxd8ttyeAs3TrmR9Mx0+jXrx6S7JxFVNoqxY8fSb0Q/+s7qy95Te+lQqwOJ\nCYmUCy1ndcimyHn/UlJg1SrYtAm2bDEuJ04U/dqICKhRw7hUrQqRkVC5svF45cpXbleqBGXLGpfw\n8Cu3y5aFMmUgyEM9B7t8NgujVLG1HjCp4CulvgDigGil1H7gNSAM0FrriVrrBUqpO5RSfwAXgEfM\n2K6/SU5OtjoEj7JDfmcunaHPrD6kZ6YzqOUgZvSdQXBQMGDkd33161kzdA1dJndhfdp6Rs4fydQ+\nU60N2k3Hj8P8+TB9ejJTpkBq6rXLRERAixbQoAFcd92VS506UL26Ubx9mR0+m2YwpeBrrQeXYJlR\nZmxLCE96dumzJJ9Opl3Ndky+Z3Jusc+reoXqfD/oe9pPas+0LdPo3aQ3fZv1tSDa0jtyBL74Ar75\nBlavBqfzynMREdCtG3TsCK1bG5eYGCjhTqTwYd7q4QsgISHB6hA8yt/z+zH5RyZumkhoUCjT+kyj\nbGjZq57Pm1+Lai34x63/4IlFT/D4wsfp1aAX5cPKezli1zid8MMPMHmysUfvcBiPh4bCbbdB48YJ\nDB9u7Ml7qrViFX//bJpF+VLPXCmlfSkeETic2kn7ie3ZfHgzY28ey2txr5XoNR0ndWTjoY28dvNr\njI0b6/lAS+HyZZgxA959F37/3XgsOBjuugsGD4b4eKO3LvyXUqpEX9qitfaZixGOfa1YscLqEDzK\nn/ObvXW2Ziy61nu1dHpGeoHLFJTfqpRVmrHosm+W1Wln0zwcpWscDq2nTdM6JkZrMC6xsVq/+67W\nhw9fWS42NlZjDJOWi49fYmNjC3yvs2tnsTVWWjoi4GU5s3hlxSsAvNL9lWtaOUW5qe5N9Gnah293\nfsu/fv4X7/Z611NhuuSXX2DkSNi82bjfqhU8/zz072+0cPJKSUmx9eg4OynpaJxCX+9Lb7S0dIQV\nvvztSwZ/PZjrIq9j52M7CQ0OLf5FeWxM20j7Se0pH1qelL+mEF0u2kORFi89HV5+GcaNM/bpa9eG\nv/8dHnyw8L58djvAu4GKUinsvSppS8dmX80I4RqtNf/65V8APHvjsy4Xe4B2tdoR3zCeC5kXGL92\nvNkhltivv0KbNvD++0Zxf+45o2c/ZIj9voQVpSMfAy9KTEy0OgSP8sf8ftr/ExvSNlClXBWGtB5S\n5LJF5fdSt5cA+Gj9R1zKumRmiCUybRp07gy7d0PLlrB2Lbz9NpSzxzFhwiRS8EVAe+/n9wAY2X6k\nS737/LrGdKVtzbacuHiCWVtnmRVesZxOePppSEiAixfhkUdg3Tpo185rIfiVoKAg9u7da3UYlpGC\n70VxcXFWh+BR/pZf6plUvv/9e8KCwxjZYWSxyxeVn1KKxzo8BsB/1v/HrBCLdPmyMazy/feNL2In\nTYIpU4xpCuykfv36LF++3JR1ufulp7+Tgi8C1pTNU3BqJ32b9qV6BfdPzzCw5UAiwyNZn7aedQfX\nmRBh4S5dgt69YdYsqFgRFi6EYcM8uklbCPQvp6Xge5E/9rhd4U/5OZwOJm+eDMCwtiWrlMXlVy60\nHENvGArAp5s+dSu+omRkwH33weLFxkRlK1dCz54e25ylhgwZwv79+7nrrruoVKkS//znPxkwYAA1\na9YkMjKSuLg4tm/fnrv8I488wqhRo3KX79KlC/v27btqnUuXLqVx48ZER0czalRgzfgiBV8EpCV7\nlpB6NpX6EfXpUb+HaevNKfizts3iYuZF09abw+Ew2jjz50NUFCxbZozM8RSlzLuUxvTp06lbty7z\n58/n7NmzjB49mjvuuIM9e/Zw9OhR2rZtywMPPHDVa2bOnMnrr7/O6dOnadCgAS+99NJVz8+fP5+N\nGzeSlJTE7NmzWbJkSWl/PH5HCr4X+VuP21X+lN+nm4098GFthxGkSvZrUJL8mldtTodaHTh7+Szf\n7vzWnRAL9Nxz8NVXxnTDS5caB1QFgrytmISEBMqVK0doaCivvvoqW7Zs4dy5c7nP9+vXj3bt2hEU\nFMQDDzxAUlLSVet64YUXqFixIjExMdxyyy3XPG9nUvBFwDl96TQ//P4DQSqIhDYJpq8/Z51Tt0w1\ndb2TJsF770FIiDHLZdu2pq6+QFcmZXD/Ygan08nzzz9Pw4YNiYiIoH79+iilOH78eO4yNWrUyL1d\nrlw5zp8/f9U6qlevXuTzdiYF34v8qcddGv6S33e7viPDkcHNsTdTq2KtEr+upPkNbDmQsOAwlu5Z\nyoGzB0oZ5dVWrzamSgCYMAFuucWU1fqFvCNrvvjiC77//nuWL1/O6dOnSU5OzjsXlyiGFHwRcGZv\nmw3AgBYDPLL+qLJR9G7SG41mxpYZbq/vxAkYNAiysowx90OHmhCkH6levXru2Plz585RpkwZIiMj\nuXDhAi+88ELAD7V0hRR8L/KnHndp+EN+py6eYsmeJQSpIPo16+fSa13J7+HWDwPw+W+fu7SN/LQ2\nDqZKTTWOpH37bbdW55deeOEF3njjDaKiojh16hSxsbHUrl2bli1bcuONN7q0rvx/HALtj4VMniYC\nytSkqTwy7xF61O/BsiHLPLadDEcGNd+rycmLJ/ltxG+0rNayVOv5z39g1CjjLFRJSRAba3KgyORp\n/kQmT/Mj/tLjLi1/yG/O9jkADGjuejvHlfzCgsPo19T4D6K0Uy0kJxujcsD4wtYTxV4EFin4ImC4\n084pjYEtBwIwc9tMl/egtYY//xkuXDDmsL/vPk9EKAKNFHwv8ocetzt8Pb9vd35LljOLHvV7ULV8\nVZdf72p+cfXiqF6+On+c/IPNhze79Npp04xx9lFR8OGHLr1UiEJJwRcBY/b27NE5pWjnlEZwUDD3\nNTd2zWdunVni150+Dc8+a9weNw6quz/NjxCAFHyv8ocetzt8Ob8T6Sf4397/EayC6dusb6nWUZr8\ncto6s7bNKnFb54034Ngx6NbNOFOVEGaRgi8CQt52TpVyVby23RtjbqROpTrsP7OfXw78Uuzyu3bB\nBx8Yc8+MH1/6OWiEKIgUfC/y9R63u3w5v9x2jhsHW5UmvyAVlNtCmrWt+NE6o0cbB1gNGwY33ODy\n5oQokhR8YXsn0k+wbO8yo53TtHTtHHfk/JGZu30uTu0sdLnVq+GHH4z57d9801vRiUAiBd+LfLnH\nbQZfze+bnd/g0A5uve5WostFl3o9pc2vY+2O1K1cl4PnDvJz6s+FLvfKK8b1U09BtWql2pQQRZKC\nL2zP03PnFEcpldvWyYklv+XLYcUK44jap57yZnSBxYxz2qakpBAUFITTWfh/a0X54osviI+PdyuG\n0pKC70W+3OM2gy/md+zCMZbvW05IUAh9mvZxa13u5Ne/RX8A5u64tq2j9ZW9+zFjjKIvrvDFc9q6\ns57BgwezaNEiU+JwlRR8YWt52zlRZaMsi6NDrQ7EVo4l7Vwaa1LXXPXc8uWwZg1UqQJPPGFRgAHC\n6jmDHA6HpduXgu9FvtrjNosv5pfbzjHhYCt38lNK5baU8rd13n3XuP7rX6FChVJvwiPU68q0S2n4\nwjltnU4no0ePpmrVqjRs2JD58+df9fzZs2cZNmwYtWrVIiYmhldeeSX3D8u0adO46aabePrpp4mO\njub1119n2rRpdOvWDYARI0YwZsyYq9bXp08fxo0bV6qfV3Gk4AvbOnrhKCuSVxAaFOp2O8cM/Ztn\nt3XyjNbZssU4GXn58jBihJXR+SZfOKftxIkTWbBgAVu2bGHDhg3MnTv3queHDBlCWFgYe/fuZfPm\nzSxdupRPP71yEvu1a9fSsGFDjh07lhtLTkto8ODBzJ59ZQfg9OnTLF26lEGDBrn+wyqJnLPF+MLF\nCEcIc3yy/hPNWPQdn99hdShaa62dTqeuN66eZix6ZfJKrbXWDz5onADwySeti8vXf+/q1aunly1b\nVuBzp06d0kopffbsWa211gkJCXr48OG5zy9YsEA3a9Ys975SSq9Zsyb3/oABA/Q777xT5PZ79Oih\nJ0yYkHt/yZIlOigoSDscDn348GFdpkwZfenSpdznv/zyS33LLbdorbWeOnWqjo2NvWp9U6dO1d26\ndcu9Hxsbq1etWqW11nrSpEm6Z8+ehcZS2HuV/XixNVb28IVteXvunOLkH62zfz98+SUEB8vInJKy\n4py2aWlpxMTE5N6PzTNP9f79+8nMzKRmzZpERUURGRnJo48+elU8eV9bkPvvv58vv/wSMEbw5P+P\nxUxS8L3IF3vcZvKl/I6cP0JiciKhQaH0btrblHWakV/e0Tr//siBwwEDBshc90Wx+py2NWvWJDU1\nNfd+SkpK7u2YmBjCw8M5ceIEJ0+e5NSpU5w+fZpff/21wPgLMmjQIObOncv+/ftZu3Yt9957r/lJ\nZJOCL2zp6x1f49ROejXoRUS474xzbFezHfUj6nP4/GEmLlwNyMic4lh9TtsBAwbwwQcfcPDgQU6d\nOsU777yT+1yNGjXo1asXTz31FOfOnUNrzd69e1m5cmWJ19+mTRuqVKnCsGHDiI+Pp1KlSp5IA5CC\n71W+OE7dTL6Unxlz5+RnRn55R+ucqTObG26ATp3cXq2tWX1O2+HDh3P77bfTunVr2rdvf80e+PTp\n08nIyKB58+ZERUXRv39/Dh8+7FJcgwYNYtmyZR5t54Cc01bY0OHzh6n1Xi1Cg0M5MvqIT+3hA2xM\n20j7Se3hfHUmND7In4cHWxqPnNPWf8g5bf2IL/W4PcFX8vt6x9doNLc3uN3UYm9WfupwWzh5HVQ4\nQt1uP5myTiFKQgq+sB2r584pzsSJCrYZsX2/p+C5dYR3jRgxgooVK1KpUiUqVaqUe3vkyJFWh2Yq\naekIWzl07hC1/1WbsOAwjo45SqUynvsCrDQuXoSaNeFMuU3wl3ZUK1+NtKfTCA6yrq0jLR3/IS0d\nIfL4asdXaDTxDeN9rtgDzJsHZ85A21o30CCyAUcvHGVlSslHdAjhDin4XuQrPW5P8YX8cs4qlTON\ngZnMyG/aNOP6kYQro3XmbJ/j9nqFKAkp+MI2Dpw9wE/7fyI8JJx7mtxjdTjXOHgQliyB0FAYNOjK\nH6WvdnxFljPLsrhiY2NRSsnFDy6xbh6hF2LSZ0aUgC+NU/cEq/PL+bL2zkZ3UrFMRdPX725+//0v\nOJ3Qpw9ER0OUbkPDqIb8cfIPVqaspEf9HuYE6qLk5GRLtiu8T/bwhW3M3DoTgIEtB1ocybW0vtLO\nSUgwrvPOrTNnm7R1hOdJwfciX+hxe5KV+e09tZf1aeupEFaBOxrd4ZFtuJPfpk2wYwdUrQp5z26X\nM7eO1W0dsPfn0865ucKUgq+UildK7VRK/a6Ueq6A5x9WSh1VSm3Kvgw1Y7tC5Ji11fiy9p4m91Au\ntJzF0VxrlhEeAwYYPfwcrau3plFUI46lH+PH5B+tCU4EDLcLvlIqCPg3cDvQAhiklGpawKIztdZt\nsy9T3N2uP7K6x+1pVuY3c1t2O6eF59o5pc1Pa8g5x8X991/9nFK+M1rHzp9PO+fmCjP28DsCu7XW\nKVrrTGAmUNB8tJ6d0k4ErB3HdvDrkV+JCI+gV4NeVodzjXXrICUFatWCrl2vfd5XRusI+zOj4NcG\nUvPcP5D9WH79lFJJSqnZSqk6JmzX79i9j2hVfjlj7/s17UeZkDIe205p88tp5/TvD0EF/MZdX/16\nGkc35nj6cRKTS7cNM9j582nn3FxhxrDMgvbc8x/7+x3whdY6Uyn1F2Aa0LOglSUkJFCvXj0AIiIi\naNOmTe6/Yzlvmr/eT0pK8ql47JCf1jq3ndP0fFMSExN9Kj+nE+bMMe43apRIYmLByw9oPoA3p7/J\n+JnjufXFWz328zI7P7lvzf3ExESmTp0KkFsvS8LtuXSUUp2BsVrr+Oz7z2OcX/GdQpYPAk5qra+Z\nxlDJXDrCResPrqfjpx2pWq4qac+kERLkW4eWrF4NN90EdetCcjIUNv36r0d+pfUnrYkuG83h0Yd9\nLg/h27w5l856oKFSKlYpFQYMxNijzxtMjTx3ewPbTdiuEEzbYgxuf6DVAz5ZJHO+rB0woPBiD9Cq\nWiuaRDfhxMUTrNi3wjvBiYDjdsHXWjuAUcASYBvGaJwdSqnXlVJ3ZS/2hFJqq1Jqc/ayCe5u1x/l\n/EtmV97OL8ORwZdbjZM/P9zmYY9vz9X8tIZvvjFu33df0cvmHa3z+W+flyI699n582nn3Fxhyjh8\nrfUirXUTrXUjrfXb2Y+9prX+Ifv2i1rrllrrG7TWPbXWv5uxXRHY5v8+n5MXT3J99etpU6ON1eFc\nIykJUlON6ZA7dCh++Yeufwgwhmeeu3zOw9GJQCRH2npRzpcvduXt/HLaOUOuH+KV7bma37x5xvU9\n9xQ8Oie/RtGNuKnuTaRnplsyJt/On0875+YKKfjCLx27cIz5u+cTrIJ54HrPnvi5tHIKfu+Cjkop\nxNA2xkHoUzYH5LGJwsOk4HuR3fuI3sxv5taZZDmzuL3h7dSoUKP4F5jAlfySk42WToUK0MOFSTD7\nt+hP+dDyrE5dze8nvNv5tPPn0865uUIKvvA7WmumJBl7wA+39vyXtaXxXfY4tfh4KOPCsWAVwirk\nTqg2NWmq+YGJgCbntBV+Z93BdXT6tBNVylXhwFMHPHp0bWn17AnLl8OMGfDgg669dlXKKrpP7U6t\nirXY/9f9lp7vVvgHb47DF8KrPtnwCQAJrRN8stifOgU//gjBwXBHKWZqvqnuTTSMakjauTQW/bHI\n/ABFwJKC70V27yN6I7/Tl07nnujkz+3+7PHt5VXS/BYsAIcDbr4ZoqJc345SimE3DAPgow0fub6C\nUrLz59POublCCr7wKzO2zOBi1kVuve5WGkU3sjqcAi1YYFzffXfp1/H/2v4/ygSXYeHuhew9tdec\nwETAkx6+8Btaa1p+3JLtx7Yzp/8c7mtezOGrFnA4oHp1OHECdu6EJk1Kv66Hv32Y6VumM7rLaN7t\n9a55QQrbkR6+sJ2VKSvZfmw7NSrUoHcTFwa3e9GGDUaxr18fGjd2b12PdXgMgClJU7iYedGE6ESg\nk4LvRXbvI3o6v3FrxwEwvO1wQoNDi1nafCXJb+FC4/pPfyp6srSS6Fi7I+1rtefkxZO531t4kp0/\nn3bOzRVS8IVf+OPkH8zbOY+w4LDcPV9flLfgmyEn1w/WfYC0O4W7pIcv/MLjCx7n3+v/zdA2Q5nc\ne7LV4RTo2DGjfx8aCidPQvny7q/zUtYlYsfFcvTCUZY8uITbGtzm/kqF7UgPX9jGqYunco+sfarL\nUxZHU7glS4wpkW++2ZxiDxAeEs6TnZ4E4J3VBZ5TSIgSk4LvRXbvI3oqvwkbJ5CemU6vBr1oWa2l\nR7ZREsXlZ3Y7J8eI9iOoEFaBZfuWsTFto7krz8POn0875+YKKfjCp13IuMB7P78HwOguoy2OpnBO\nJyxebNyOjzd33ZFlI/lLu78A8I81/zB35SKgSA9f+LT31rzH6KWj6VynM2uGrkG5O/TFQ9atg06d\nIDYW9u1zf4ROfgfPHqT++Po4tINdo3bRMKqhuRsQfk16+MLvpWem8+4a44CjV7u/6rPFHq7s3Zsx\nHLMgtSvV5qHrH8Kpnfztx7+ZvwEREKTge5Hd+4hm5zdp4ySOXDhC+1rtiW9ocp+kFIrKb/ly4/o2\nDw6iebn7y4QGhfLfX//L9mPbTV+/nT+fds7NFVLwhU86d/kcf//p7wC80v0Vn967v3gR1qwx9uxv\nvtlz26kfWZ9hbYeh0byW+JrnNiRsS3r4wie9uuJV3lj5Bl3qdGH10NU+XfCXLzfmv2/TBjZv9uy2\n0s6l0eCDBlzKusTGP2+kbc22nt2g8AvSwxd+K+1cWu7InHdve9eniz1caee4cirD0qpVsRYj248E\nYMzSMXL0rXCJFHwvsnsf0az8xiaOJT0znb5N+9K1bldT1mmGwvJbscK49kbBB3ip+0tElY1i+b7l\nfLvzW9PWa+fPp51zc4UUfOFTNqZtZPLmyQSrYN7q+ZbV4RTr/HljSGZwMHTr5p1tRpWN4m9xxkid\nZ5Y8w6WsS97ZsPB70sMXPsPhdNB5cmc2pG3gmS7P8M9e/7Q6pGItWmQMxezUCX75xXvbzXJmccOE\nG9h6dCv/1+P/eLHbi97buPA50sMXfueTDZ+wIW0DdSrVYWzcWKvDKZGc/v0tt3h3uyFBIYyPHw/A\nGyvfYPeJ3d4NQPglKfheZPc+ojv5pZ1L46XlLwEwPn48FcIqmBSVeQrKz5tf2ObXo34PHrr+IS5l\nXWLY98Nwaqdb67Pz59POublCCr6wnNaaR+Y9wpnLZ7ir8V30bdrX6pBK5NQpYxhmaCh0tei75fdv\nf59q5auxMmUln2z4xJoghN+QHr6w3H/W/YdRC0cRVTaKrSO2UrNiTatDKpF586BPH+PL2pUrrYvj\nq+1fcd+c+ygfWp5Nf9lE42g3z60o/I708IVf2Hl8J2OWjgFg4l0T/abYw5XhmN7u3+d3b/N7Gdhy\nIBcyL3D/3Ptl1I4olBR8L7J7H9HV/M5dPke/Wf24mHWRh65/iHub3+uZwEySPz8r+/f5TbhrAg0i\nG5B0OIkxS8aUah12/nzaOTdXSMEXlsjp2+84voPmVZvznzv+Y3VILjl2DH77DcLDoXNnq6OBSmUq\nMfO+mYQGhfLv9f/m818/tzok4YOkhy8s8daqt3hx+YtUKlOJ9cPX+13fec4cGDDAmEPnf/+zOpor\nPlr/EY8teIyw4DCWD1nuU0cqC8+RHr7wWdO3TOfF5S+iUMzoO8Pvij34Tv8+v5EdRvJYh8fIcGTQ\nd1Zf9p7aa3VIwodIwfciu/cRS5Lfwt0LGTpvKGCMt7+nyT0ejso8efPzpf59fuPix9GrQS+OpR/j\n1um3cuDsgRK9zs6fTzvn5gop+MJrFv2xiH6z++HQDp7v+jyPd3rc6pBKJS0Ndu2CChWgfXuro7lW\nSFAIc/rPoUOtDuw7vY+e03ty+Pxhq8MSPkB6+MIrvtv1Hf3n9CfDkcGj7R7lozs/8vlpjwvz+efw\n4IPGHDoLFlgdTeFOXjxJj2k92HJkC42iGrHkoSXUi6hndVjCA6SHL3zGhA0T6DerHxmODJ7s9KRf\nF3vw7XZOXlFlo1j60FJaV2/N7pO7uXHyjWw5vMXqsISFpOB7kd37iPnzy3Jm8dSip3h0/qM4tIOX\nur3E+7e/77fFPic/X/3CtiBVy1flx4QfiasXx6Hzh7jps5uYs21Ogcva+fNp59xcIQVfeETK6RTi\npsYxbu04QoNC+az3Z7zZ402/LfY5kpNh3z6IiDBOaegPKodXZtEDixjcajDnM84zYO4Anl78NJmO\nTKtDE14mPXxhKq01M7fOZMT8EZy5fIZaFWsx896ZdIv10tlBPOyzz2DoUOjdG74172RTXqG15sN1\nH/LMkmfIcmbRpkYbPuv9GW1q+MlfLlEo6eELr/vj5B/Efx7P4K8Hc+byGe5pcg9bHt1im2IP/tO/\nL4hSiic6PUHiw4nUj6hP0uEkOkzqwMvLX+Z8xnmrwxNeIAXfi+zaRzx64SjPLH6G5mOas2TPEiLC\nI5h410S+vf9bqpSrYnV4plmxItGv+veF6Vq3K7+O+JVRHUaR5czi/1b9H40/bMyzk57F4XRYHZ5H\n2PV3z1VS8EWpHTx7kBf+9wLXjb+Of/3yLzIdmTx0/UPsGrWL4e2G+32/Pr8DB+DgQahaFVq0sDoa\n91QIq8CHd3zI6qGr6VCrA4fOH+Ld1e/S5N9NmLBhgsy4aVPSwxcucWonP+3/iY83fMzc7XPJcmYB\ncGejO/nbLX+jbc22FkfoORMmwKOPQv/+MHu21dGYx6mdzNw6k1dXvMqeU3sAqFa+GgmtExh6w1Ca\nVGlicYSiOCXt4UvBF8VyaiebD21m9rbZfLn1S1LPpgIQrILp26wvz3R5hs51fGDKSA+7/36j0H/8\nsVH47SbLmcVX27/i7dVvk3Q4KffxrjFdubfZvfRu2pvrIq+zMEJRGCn4PigxMZG4uDirwyiW1pqU\nMyms3r+axXsWs3jPYo5eOJr7fN3KdXmw1YM82v5RYirH5D7uL/mVhtYQGZnImTNx7NwJTWy405vz\n/mmtWZO6himbpzBr2ywuZF7IXaZVtVb0atCL7rHd6Va3G5FlIy2MuOTs/NmEkhf8EJM2Fg+Mw/hO\nYLLW+p18z4cB04F2wHHgfq31fjO2LdzjcDrYe2ov249tZ+vRraxLW8faA2s5cuHIVcvFVIrh7sZ3\nM7jVYLrEdCFIBdbXP9u2wZkzUKsWNPa/yT1dopSia92udK3blXHx4/jh9x+Yt2seC3Yv4Lejv/Hb\n0d947+f3UChaVmvJDTVvoE31NrSu0ZrW1VsTXS7a6hREIdzew1dKBQG/Az2BNGA9MFBrvTPPMiOA\nVlrrkUqp+4G+WuuBBazL1nv43qa1Jj0znRMXT5B2Lo39Z/aTeiaV/Wf2s//sfvae2suu47u47Lh8\nzWujy0ZIaeIMAAAUO0lEQVTTqU4netbvSXzDeJpVaWa7L2Fd8cEH8OSTxhw6M2ZYHY01LmddZtX+\nVSQmJ7IyZSVrD64lw5FxzXKR4ZE0iGpAg0jjUj+yPjUr1KRGhRrUqFCDauWrERocakEG9uW1lo5S\nqjPwmtb6T9n3nwd03r18pdSi7GXWKqWCgcNa66oFrCtgCr7WmkxnJpmOTDIcGSW6nZ6ZzvmM81zI\nvGBcZ1y46vbZjLOcSD/B8fTjHE8/zomLJ0o02iKmUgzNqzanWZVmtKvVjs51OtMgskFAF/j8+vY1\nDrSaPNk48ErApaxLbDq0iS2Ht7DlyBaSDiex9ejWq1pABVEoqpSrQnS5aCqXqUxEeASVwysTUSb7\nOjyC8qHlKRtalvCQcMqGlC3wdpngMoQEhRASFEJocGju7byXQPlP1JstndpAap77B4COhS2jtXYo\npU4rpaK01idN2H6pPLv0WRb+sRCndqK1RqOvuq119v1ibud/XVHryNqbhaqvcDgdOLR3xjuXCS5D\nlXJVqF6hOrGVY6lbuS4xlWKoW7ku9SLq0bRKUyqWqWjKtuzaJ3U4wBjGnUiPHnFWhuJRrr5/4SHh\n3BhzIzfG3Jj7mNaaIxeOsOfkHvae2sueU3tIOZPC4fOHcy9HLxzlWPoxjqUf80AWV1MoQoJCUMmK\nMg3LEBwUjEKhlMq9DlJBpXosSAVd9XzOdY5Vj6wiIjzC4zm6woyCX9Bflfy76fmXUQUsA0BCQgL1\n6tUDICIigjZt2uR+CHMOnjDjfurZVLau3WpstH72xvfh2fsHAceV+0EpQYQGhRLeMJzQ4FCce52E\nBodSsUlFQoNCydiTQXBQMNHNogkNDuXS7kuUDS1L/RvqUz60PCd3nCQ8OJxWnVpRPqw8qVtSqVym\nMj169CC6bDTb128nPCScW7KPEsrN/8YrP4+Nuzea9vNNSkpy6/W+er9SpThOn4bIyCSSk6FePd+K\nzxffvxoVapC5N5MYYojrffXz3bp341j6MRb/bzHnM87ToG0DTl86zbrV6zifcZ7oZtGkZ6azZ/Me\nLjsuU7lJZS5mXeTQb4e4nHWZsIZhXMy8yJmdZ3BoByHXhZDlzOLi7ovGjlQ9Y8RR5p5MNJrM+png\ngIxd2e0nL/2+r/xxJZXKVPLI+5WYmMjUqVMBcutlSZjV0hmrtY7Pvl9QS2dh9jI5LZ1DWutqBazL\nay2dlNMpnL18ttC/1nn/mhf1l72w20WtI0gZhV5aJv7hn/+EMWOMVs7kyVZHI1zh1E6ynFlXXfL+\nB17Yf+0lfSz/f/R5NavajJAgU8bFFMubLZ31QEOlVCxwCBgIDMq3zPfAw8BaoD+w3ITtuiU2Itbq\nEISfsMN0CoEqSAURFhxGWHCY1aH4BLe/0dBaO4BRwBJgGzBTa71DKfW6Uuqu7MUmA1WUUruBvwLP\nu7tdf2T3+TzsmF9mJqxcadwuUybR0lg8zY7vXw475+YKU/7f0FovAprke+y1PLcvAwPM2JYQ3rRx\nI5w/b4y9r3rNuDIh/IscaStEEf7+d3jpJWMqhY8/tjoaIQom8+ELYYKc+e+lfy/sQAq+F9m9j2i3\n/C5fhtWrjdtxcfbLLz8752fn3FwhBV+IQvzyC1y6BC1bQrVrBhEL4X+khy9EIcaOhddfhyeegPHj\nrY5GiMJJD18IN/nz+WuFKIgUfC+yex/RTvmlpxstHaWge3fjMTvlVxA752fn3FwhBV+IAqxebRx0\n1bYtRPrHOT6EKJb08IUowIsvwltvwejR8O67VkcjRNGkhy+EG2T8vbAjKfheZPc+ol3yO3MG1q+H\nkJAr/XuwT36FsXN+ds7NFVLwhcjnxx/B6YROnaBCBaujEcI80sMXIp+//tUYd//KK/C3v1kdjRDF\nkx6+EKW0bJlx3bOntXEIYTYp+F5k9z6iHfI7cgS2boWyZaFz56ufs0N+RbFzfnbOzRVS8IXII+fs\nVjfdBGXKWBuLEGaTHr4QeQwfDp9+Cm+/Dc89Z3U0QpSM9PCFKAXp3ws7k4LvRXbvI/p7fvv2GZeI\nCLjhhmuf9/f8imPn/Oycmyuk4AuRLefo2rg4CA62NBQhPEJ6+EJkGzwYvvwSPvwQRo2yOhohSk56\n+EK4QOsre/jSvxd2JQXfi+zeR/Tn/LZvN8bg16wJTZsWvIw/51cSds7Pzrm5Qgq+EFwZndOjh3HS\nEyHsSHr4QgC9e8N338GUKfDII1ZHI4RrStrDl4IvAl5GBkRHw/nzkJoKdepYHZEQrpEvbX2Q3fuI\n/prfzz8bxb5586KLvb/mV1J2zs/OublCCr4IeIsXG9e3325tHEJ4mrR0RMBr1w42bYJFi6ToC/8k\nPXwhSuDoUaheHcLD4eRJY1pkIfyN9PB9kN37iP6Y39KlxnX37sUXe3/MzxV2zs/OublCCr4IaIsW\nGdfSyhGBQFo6ImA5ncaRtUePGme5atHC6oiEKB1p6QhRjC1bjGJfu7YxJFMIu5OC70V27yP6W345\nwzHj40s2nYK/5ecqO+dn59xcIQVfBCzp34tAIz18EZBOnYKqVY3bx48bZ7kSwl9JD1+IIixeDA4H\ndOsmxV4EDin4XmT3PqI/5ff998b13XeX/DX+lF9p2Dk/O+fmCin4IuBkZcHChcbtu+6yNhYhvEl6\n+CLgrFwJN98MjRvDrl1WRyOE+6SHL0Qhcto5sncvAo0UfC+yex/RX/L74Qfj2pX+PfhPfqVl5/zs\nnJsrpOCLgPLHH7BzJ1SuDF27Wh2NEN4lPXwRUN5/H55+GgYOhC+/tDoaIcwhPXwhCvDNN8b1PfdY\nG4cQVpCC70V27yP6en6HD8NPP0FYGNx5p+uv9/X83GXn/OycmyvcKvhKqUil1BKl1C6l1GKlVOVC\nlnMopTYppTYrpb51Z5tClNY334DWxtw5lSpZHY0Q3udWD18p9Q5wQmv9D6XUc0Ck1vr5ApY7q7Uu\n9ldMevjCk269FZYtg6lT4eGHrY5GCPN45Zy2SqmdwM1a6yNKqRpAota6aQHLndNaVyzB+qTgC484\nfhxq1DCmQT56FCIjrY5ICPN460vbalrrIwBa68NA1UKWK6OUWqeUWqOU6u3mNv2W3fuIvpzfvHnG\nZGk9e5a+2Ptyfmawc352zs0VIcUtoJRaClTP+xCggZdd2E5drfVhpVR9YLlS6let9b6CFkxISKBe\nvXoARERE0KZNG+Li4oArb5q/3k9KSvKpeAIpv7lzARJp2RLAfvmZcd/u+dnpfmJiIlOnTgXIrZcl\n4W5LZwcQl6els0Jr3ayY13wGfK+1/rqA56SlI0x38qTRznE4jJE6VQv7P1QIP+Wtls53QEL27YeB\neQUEEqGUCsu+XQW4Edju5naFKLG5cyEzE3r0kGIvApu7Bf8d4Dal1C7gVuBtAKVUO6XUxOxlmgEb\nlFKbgWXAW1rrnW5u1y/l/EtmV76a33//a1w/+KB76/HV/Mxi5/zsnJsriu3hF0VrfRKj0Od/fCPw\n5+zbPwPXu7MdIUorJQVWrYKyZaFvX6ujEcJaMpeOsLW334YXXoD774eZM62ORgjPkLl0RMDT2rx2\njhB2IAXfi+zeR/S1/H79FbZtg+hoYzoFd/lafmazc352zs0VUvCFbc2YYVzffz+EhlobixC+QHr4\nwpYyMqB2bWNKhV9+gU6drI5ICM+RHr4IaPPmGcW+VSvo2NHqaITwDVLwvcjufURfym/SJON6+HBj\nwjQz+FJ+nmDn/Oycmyuk4Avb2bcPli6F8HAZnSNEXtLDF7bzyivw5ptGsc/54lYIO/PKfPhmk4Iv\n3HX5MtSrZ0ySlpgIN99sdURCeJ58aeuD7N5H9IX8Zs82in2rVtC9u7nr9oX8PMnO+dk5N1dIwRe2\noTWMG2fcfvJJ876sFcIupKUjbOOnn6BbN6hSBfbvNyZMEyIQSEtHBJzx443rRx+VYi9EQaTge5Hd\n+4hW5vfHH/D11xASAiNGeGYb8v75Lzvn5gop+MIW3n4bnE4YMgRq1bI6GiF8k/Twhd/bvx8aNDAK\n/s6d0KiR1REJ4V3SwxcB4x//gKwsGDhQir0QRZGC70V27yNakV9aGnz6qXH7xRc9uy15//yXnXNz\nhRR84ddef904uvbee6FFC6ujEcK3SQ9f+K0dO4wjasE4s1WTJtbGI4RVpIcvbO+FF8DhgGHDpNgL\nURJS8L3I7n1Eb+b300/GSU7KlYPXXvPONuX98192zs0VUvCF38nKglGjjNujR0PNmtbGI4S/kB6+\n8Dvvvw9PP21Mg7xtm7GXL0Qgk/nwhS0dOADNmsH58/D993DXXVZHJIT15EtbH2T3PqKn89PaaOWc\nPw99+3q/2Mv757/snJsrpOALvzFtmvFFbaVKV2bGFEKUnLR0hF/Ytw9at4Zz52D6dHjoIasjEsJ3\nSEtH2EZmpjEL5rlzcN99xsnJhRCuk4LvRXbvI3oqvzFjjHH3NWvCJ59Yd+pCef/8l51zc4UUfOHT\n/vtfo18fGgpffQXR0VZHJIT/kh6+8Fm//AI9esDFi/Dxx8apC4UQ15IevvBrO3bAnXcaxX74cPjL\nX6yOSAj/JwXfi+zeRzQrv9RUiI+HkyeNsfYffWRd3z4vef/8l51zc4UUfOFT9u6F7t2N0xZ26QKz\nZhknJhdCuE96+MJn7NwJt94KBw9Cp06wcCFERlodlRC+T3r4wq8sXWrs0R88aOzhL10qxV4Is0nB\n9yK79xFLk5/W8MEH8Kc/wenT0KePsWdfsaL58blL3j//ZefcXCEFX1jm2DHo3RuefNI4c9WLLxpj\n7WW6YyE8Q3r4wuu0hq+/hscfh0OHICICJk6E/v2tjkwI/1TSHr6MfxBetXs3PPEELFpk3O/eHWbM\ngLp1rY1LiEAgLR0vsnsfsaj8UlKMk403a2YU+4gIY3z98uX+U+wD+f3zd3bOzRWyhy88atMm+PBD\n+PxzY9bL4GAYOhTeeguqVbM6OiECi/TwhelOnIC5c40Tlvz8s/GYUvDAA/Dqq9CokbXxCWE30sMX\nXrV3LyxZYpxndskSyMoyHo+IMPboR4yAhg2tjVGIQOdWD18pdZ9SaqtSyqGUalvEcvFKqZ1Kqd+V\nUs+5s01/Zpc+otMJ27fD1KnGDJaNGkGDBjBiRCILFhijcG6/3Xj+wAF47z17FHu7vH+FsXN+ds7N\nFe7u4f8G9AUmFLaAUioI+DfQE0gD1iul5mmtd7q5bb+TlJREXFyc1WGUWFaWceTrzp3GZccO47J5\ns3H2qbwiIqBu3SRGjYqjTx+oWtWamD3J394/V9k5Pzvn5gq3Cr7WeheAUkXOZdgR2K21TsledibQ\nGwi4gn/69GmrQyAjA06dMvrsJ09efX3okDFTZWqqsWd+6JBxQFRB6taFjh2Ny003QYcO8Oabpxk+\n3Lv5eJMvvH+eZOf87JybK7zRw68NpOa5fwDjj4Cl9uyB48eN9kNRFyh+mZKuY8cO44Cj4pbPyjIu\nmZnGJe/t/Pfz3r50CdLT4cKFK9d5b6enG8u4okYNaNLEGE7ZrBk0bQrXX288LoTwL8UWfKXUUqB6\n3ocADbyktf6+BNsoaO/f8qE4L78MM2d6e6vJzJ7t7W1eLTgYoqKuXKKjr9yuWRPq1IGYGONSqxaE\nhZV83cnJyR6L2xdIfv7Lzrm5wpRhmUqpFcAzWutNBTzXGRirtY7Pvv88oLXW7xSwrOV/CIQQwh95\ne1hmYRtbDzRUSsUCh4CBwKCCFixJwEIIIUrH3WGZfZRSqUBn4Ael1MLsx2sqpX4A0Fo7gFHAEmAb\nMFNrvcO9sIUQQrjKp460FUII4Tk+OXmaUurx7AO1flNKvW11PJ6glBqtlHIqpaKsjsVMSql/KKV2\nKKWSlFJfKaUqWR2Tu+x84KBSqo5SarlSanv279sTVsfkCUqpIKXUJqXUd1bHYjalVGWl1Jzs37tt\nSqlOhS3rcwVfKRUH3A201Fq3Av5pbUTmU0rVAW4FUqyOxQOWAC201m2A3cALFsfjljwHDt4OtAAG\nKaWaWhuVqbKAp7XWzYEuwGM2yy/Hk8B2q4PwkPHAAq11M6A1UGjL3OcKPjACeFtrnQWgtT5ucTye\n8D4wxuogPEFr/T+ttTP77i9AHSvjMUHugYNa60wg58BBW9BaH9ZaJ2XfPo9RLGpbG5W5snew7gA+\ntToWsymlKgLdtNafAWits7TWZwtb3hcLfmOgu1LqF6XUCqVUe6sDMpNS6m4gVWv9m9WxeMFQYKHV\nQbipoAMHbVUQcyil6gFtgLXWRmK6nB0sO35heR1wXCn1WXbLaqJSqmxhC1syW2YRB3O9nB1ThNa6\ns1KqAzAbIym/UUx+LwK35XvOr5TkYDyl1EtAptb6CwtCNJNPHjhoNqVUBWAu8GT2nr4tKKXuBI5o\nrZOy28V+9/tWjBCgLfCY1nqDUmoc8DzwWmELe53W+rbCnlNKPQp8nb3c+uwvNqO11ie8FqCbCstP\nKdUSqAdsyZ5/qA6wUSnVUWt91IshuqWo9w9AKfUwxr/QPbwTkUcdAPKek6sOxiSAtqGUCsEo9jO0\n1vOsjsdkXYF7lFJ3AGWBikqp6VrrIRbHZZYDGB2DDdn35wKFDizwxZbOtxgza6KUagyE+lOxL4rW\neqvWuobW+jqtdX2MN+sGfyr2xVFKxQPPAvdorS9bHY8Jcg8cVEqFYRw4aLeRHlOA7Vrr8VYHYjat\n9Yta67pa6+sw3rvlNir2aK2PAKnZtRKM2lnol9O+eAKUz4ApSqnfgMuAbd6cAmjs9y/mh0AYsDR7\nEtVftNYjrQ2p9LTWDqVUzoGDQcBkOx04qJTqCjwA/KaU2ozxmXxRa73I2siEC54APldKhQJ7gUcK\nW1AOvBJCiADhiy0dIYQQHiAFXwghAoQUfCGECBBS8IUQIkBIwRdCiAAhBV8IIQKEFHwhhAgQUvCF\nECJA/H+tGwXMVYJT/AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fea5d453490>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "testValues = np.arange(-5,5,0.01)\n",
    "plt.plot(testValues, tanh(testValues), linewidth=2)\n",
    "plt.plot(testValues, tanh_deriv(testValues), linewidth=2)\n",
    "plt.grid(1)\n",
    "plt.legend(['tanh', 'tanh_deriv'])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now replace dyHat/dz3 with f prime of z 3."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\frac{\\partial J}{\\partial W^{(2)}}= \n",
    "-(y-\\hat{y}) f^\\prime(z^{(3)}) \\frac{\\partial z^{(3)}}{\\partial W^{(2)}}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our final piece of the puzzle is dz3dW2, this term represents the change of z, our third layer activity, with respect to the weights in the second layer."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Z three is the matrix product of our activities, a two, and our weights, w two. The activities from layer two are multiplied by their correspond weights and added together to yield z3. If we focus on a single synapse for a moment, we see a simple linear relationship between W and z, where a is the slope. So for each synapse, dz/dW(2) is just the activation, a on that synapse!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "z^{(3)} = a^{(2)}W^{(2)} \\tag{3}\\\\\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another way to think about what the calculus is doing here is that it is “backpropagating” the error to each weight, by multiplying by the activity on each synapses, the weights that contribute more to the error will have larger activations, and yield larger dJ/dW2 values, and those weights will be changed more when we perform gradient descent. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We need to be careful with our dimensionality here, and if we’re clever, we can take care of that summation we got rid of earlier. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first part of our equation, y minus yHat is of the same dimension as our output data, 3 by 1. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "F prime of z three is of the same size, 3 by 1, and our first operation is scalar multiplication. Our resulting 3 by 1 matrix is referred to as the backpropagating error, delta 3."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We determined that dz3/dW2 is equal to the activity of each synapse. Each value in delta 3 needs to be multiplied by each activity. We can achieve this by transposing a2 and matrix multiplying by delta3. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\frac{\\partial J}{\\partial W^{(2)}} = \n",
    "(a^{(2)})^T\\delta^{(3)}\\tag{6}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\delta^{(3)} = -(y-\\hat{y}) f^\\prime(z^{(3)}) \n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another way to think about what’s happening here is that is that each example our algorithm sees has a certain cost and a certain gradient. The gradient with respect to each example pulls our gradient descent algorithm in a certain direction. It's like every example gets a vote on which way is downhill, and when we perform batch gradient descent we just add together everyone’s vote, call it downhill, and move in that direction."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Part of NN Class (won't work alone, needs to be included in class as \n",
    "# shown in below and in partFour.py):\n",
    "\n",
    "def costFunctionPrime(self, X, y):\n",
    "    #Compute derivative with respect to W and W2 for a given X and y:\n",
    "    self.yHat = self.forward(X)\n",
    "\n",
    "    delta3 = np.multiply(-(y-self.yHat), self.sigmoidPrime(self.z3))\n",
    "    dJdW2 = np.dot(self.a2.T, delta3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have one final term to compute: dJ/dW1. The derivation begins the same way, computing the derivative through our final layer: first dJ/dyHat, then dyHat/dz3, and we called these two taken together form our backpropagating error, delta3. We now take the derivative “across” our synapses, this is a little different from out job last time, computing the derivative with respect to the weights on our synapses. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\frac{\\partial J}{\\partial W^{(1)}} = (y-\\hat{y})\n",
    "\\frac{\\partial \\hat{y}}{\\partial W^{(1)}}\n",
    "$$\n",
    "\n",
    "$$\n",
    "\\frac{\\partial J}{\\partial W^{(1)}} = (y-\\hat{y})\n",
    "\\frac{\\partial \\hat{y}}{\\partial z^{(3)}}\n",
    "\\frac{\\partial z^{(3)}}{\\partial W^{(1)}}\n",
    "$$\n",
    "\n",
    "$$\n",
    "\\frac{\\partial J}{\\partial W^{(1)}} = -(y-\\hat{y}) f^\\prime(z^{(3)}) \\frac{\\partial z^{(3)}}{\\partial W^{(1)}}\n",
    "$$\n",
    "\n",
    "$$\n",
    "\\frac{\\partial z^{(3)}}{\\partial W^{(1)}} = \\frac{\\partial z^{(3)}}{\\partial a^{(2)}}\\frac{\\partial a^{(2)}}{\\partial W^{(1)}}\n",
    "$$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There’s still a nice linear relationship along each synapse, but now we’re interested in the rate of change of z(3) with respect to a(2). Now the slope is just equal to the weight value for that synapse. We can achieve this mathematically by multiplying by W(2) transpose. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\frac{\\partial J}{\\partial W^{(1)}} = \\delta^{(3)} \n",
    "(W^{(2)})^{T}\n",
    "\\frac{\\partial a^{(2)}}{\\partial W^{(1)}}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\frac{\\partial J}{\\partial W^{(1)}} = \\delta^{(3)} \n",
    "(W^{(2)})^{T}\n",
    "\\frac{\\partial a^{(2)}}{\\partial z^{(2)}}\n",
    "\\frac{\\partial z^{(2)}}{\\partial W^{(1)}}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our next term to work on is da(2)/dz(2) – this step is just like the derivative across our layer 3 neurons, so we can just multiply by f prime(z2). "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\frac{\\partial J}{\\partial W^{(1)}} = \\delta^{(3)} \n",
    "(W^{(2)})^{T}\n",
    "f^\\prime(z^{(2)})\n",
    "\\frac{\\partial z^{(2)}}{\\partial W^{(1)}}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our final computation here is dz2/dW1. This is very similar to our dz3/dW2 computation, there is a simple linear relationship on the synapses between z2 and w1, in this case though, the slope is the input value, X. We can use the same technique as last time by multiplying by X transpose, effectively applying the derivative and adding our dJ/dW1’s together across all our examples. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\frac{\\partial J}{\\partial W^{(1)}} = \n",
    "X^{T}\n",
    "\\delta^{(3)} \n",
    "(W^{(2)})^{T}\n",
    "f^\\prime(z^{(2)})\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\frac{\\partial J}{\\partial W^{(1)}} = \n",
    "X^{T}\\delta^{(2)} \\tag{7}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\delta^{(2)} = \\delta^{(3)} \n",
    "(W^{(2)})^{T}\n",
    "f^\\prime(z^{(2)})\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "All that’s left is to code this equation up in python. What’s cool here is that if we want to make a deeper neural network, we could just stack a bunch of these operations together. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class Neural_Network(object):\n",
    "    def __init__(self):        \n",
    "        #Define Hyperparameters\n",
    "        self.inputLayerSize = 2\n",
    "        self.outputLayerSize = 1\n",
    "        self.hiddenLayerSize = 3\n",
    "        \n",
    "        #Weights (parameters)\n",
    "        self.W1 = np.random.randn(self.inputLayerSize,self.hiddenLayerSize)\n",
    "        self.W2 = np.random.randn(self.hiddenLayerSize,self.outputLayerSize)\n",
    "        \n",
    "    def forward(self, X):\n",
    "        #Propogate inputs though network\n",
    "        self.z2 = np.dot(X, self.W1)\n",
    "        self.a2 = self.sigmoid(self.z2)\n",
    "        self.z3 = np.dot(self.a2, self.W2)\n",
    "        yHat = self.sigmoid(self.z3) \n",
    "        return yHat\n",
    "        \n",
    "    def sigmoid(self, z):\n",
    "        #Apply sigmoid activation function to scalar, vector, or matrix\n",
    "        return 1/(1+np.exp(-z))\n",
    "    \n",
    "    def sigmoidPrime(self,z):\n",
    "        #Gradient of sigmoid\n",
    "        return np.exp(-z)/((1+np.exp(-z))**2)\n",
    "    \n",
    "    def costFunction(self, X, y):\n",
    "        #Compute cost for given X,y, use weights already stored in class.\n",
    "        self.yHat = self.forward(X)\n",
    "        J = 0.5*sum((y-self.yHat)**2)\n",
    "        return J\n",
    "        \n",
    "    def costFunctionPrime(self, X, y):\n",
    "        #Compute derivative with respect to W and W2 for a given X and y:\n",
    "        self.yHat = self.forward(X)\n",
    "        \n",
    "        delta3 = np.multiply(-(y-self.yHat), self.sigmoidPrime(self.z3))\n",
    "        dJdW2 = np.dot(self.a2.T, delta3)\n",
    "        \n",
    "        delta2 = np.dot(delta3, self.W2.T)*self.sigmoidPrime(self.z2)\n",
    "        dJdW1 = np.dot(X.T, delta2)  \n",
    "        \n",
    "        return dJdW1, dJdW2\n",
    "    \n",
    "    #Helper Functions for interacting with other classes:\n",
    "    def getParams(self):\n",
    "        #Get W1 and W2 unrolled into vector:\n",
    "        params = np.concatenate((self.W1.ravel(), self.W2.ravel()))\n",
    "        return params\n",
    "    \n",
    "    def setParams(self, params):\n",
    "        #Set W1 and W2 using single paramater vector.\n",
    "        W1_start = 0\n",
    "        W1_end = self.hiddenLayerSize * self.inputLayerSize\n",
    "        self.W1 = np.reshape(params[W1_start:W1_end], (self.inputLayerSize , self.hiddenLayerSize))\n",
    "        W2_end = W1_end + self.hiddenLayerSize*self.outputLayerSize\n",
    "        self.W2 = np.reshape(params[W1_end:W2_end], (self.hiddenLayerSize, self.outputLayerSize))\n",
    "        \n",
    "    def computeGradients(self, X, y):\n",
    "        dJdW1, dJdW2 = self.costFunctionPrime(X, y)\n",
    "        return np.concatenate((dJdW1.ravel(), dJdW2.ravel()))\n",
    "        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "NN=Neural_Network()\n",
    "cost1 = NN.costFunction(X,y)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.00881479])"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cost1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "cost1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "dJdW1, dJdW2 = NN.costFunctionPrime(X,y)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "scalar = 3\n",
    "NN.W1 = NN.W1 + scalar*dJdW1\n",
    "NN.W2 = NN.W2 + scalar*dJdW2\n",
    "cost2 = NN.costFunction(X,y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So far we’ve built a neural network in python, computed a cost function to let us know how well our network is performing, computed the gradient of our cost function so we can train our network, and last time we numerically validated our gradient computations. After all that work, it’s finally time to train our neural network. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from scipy import optimize"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class trainer(object):\n",
    "    def __init__(self, N):\n",
    "        #Make Local reference to network:\n",
    "        self.N = N\n",
    "        \n",
    "    def callbackF(self, params):\n",
    "        self.N.setParams(params)\n",
    "        self.J.append(self.N.costFunction(self.X, self.y))   \n",
    "        \n",
    "    def costFunctionWrapper(self, params, X, y):\n",
    "        self.N.setParams(params)\n",
    "        cost = self.N.costFunction(X, y)\n",
    "        grad = self.N.computeGradients(X,y)\n",
    "        \n",
    "        return cost, grad\n",
    "        \n",
    "    def train(self, X, y):\n",
    "        #Make an internal variable for the callback function:\n",
    "        self.X = X\n",
    "        self.y = y\n",
    "\n",
    "        #Make empty list to store costs:\n",
    "        self.J = []\n",
    "        \n",
    "        params0 = self.N.getParams()\n",
    "\n",
    "        options = {'maxiter': 200, 'disp' : True}\n",
    "        _res = optimize.minimize(self.costFunctionWrapper, params0, jac=True, method='BFGS', \\\n",
    "                                 args=(X, y), options=options, callback=self.callbackF)\n",
    "\n",
    "        self.N.setParams(_res.x)\n",
    "        self.optimizationResults = _res"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we plot the cost against the number of iterations through training, we should see a nice, monotonically decreasing function. Further, we see that the number of function evaluations required to find the solution is less than 100, and far less than the 10^27 function evaluation that would have been required to find a solution by brute force, as shown in part 3. Finally, we can evaluate our gradient at our solution and see very small values – which make sense, as our minimum should be quite flat."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "NN = Neural_Network()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "T = trainer(NN)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Optimization terminated successfully.\n",
      "         Current function value: 0.000000\n",
      "         Iterations: 49\n",
      "         Function evaluations: 55\n",
      "         Gradient evaluations: 55\n"
     ]
    }
   ],
   "source": [
    "T.train(X,y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEPCAYAAABRHfM8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X+UXWV97/H3JwlJgBscgqBtAowFJAri8MOYCtQRWxMt\nGKQKCQvJqK31UqxoLyVga0hdilGxwYtdrGpkkHKb24rIrwK5CuMPCiEBAiGEEKv5JREsMAETCCH5\n3j/2PtknhzOTM3Nmzzlz9ue11l7nPHs/e+/nfNfM+Z79PPuHIgIzM7O9GdXoBpiZ2cjghGFmZjVx\nwjAzs5o4YZiZWU2cMMzMrCZOGGZmVpPcE4akGZKekPSkpEuqLD9V0oOSdkg6q2LZAkmPSVolaWHe\nbTUzs77lmjAkjQKuBqYDxwCzJU2pqLYemAPcULHuHwLviohjgWOBqZL+KM/2mplZ38bkvP2pwNqI\nWA8gaTEwE3iiVCEiNqTLKq8gDGC8pPEkiW0M8HTO7TUzsz7k3SU1CdhYVt6UzturiLgf6AE2A78G\n7oqINUPdQDMzq03eCUNV5tV0LxJJRwBTgN8nSTLvlXTKELbNzMwGIO8uqU3AYWXlycBTNa77IeD+\niHgJQNIdwDTg5+WVqnRlmZlZDSKi2o/6PuV9hLEMOFLS4ZLGArOAW/qpX974DcC7JY2WtA/wbmB1\ntZUiwlME8+bNa3gbmmVyLBwLx6L/aTByTRgRsRO4EFgCrAIWR8RqSfMlnQ4g6SRJG4EPA9dIWpmu\n/n3gl8BK4GHg4Yi4Pc/2jnTr1q1rdBOahmORcSwyjkV98u6SIiLuBI6umDev7P1y4NAq6+0CPpV3\n+8zMrDa+0ruFdHV1NboJTcOxyDgWGceiPhpsX1azkBQj/TOYmQ03SUSTDXrbMOrp6Wl0E5qGY5Fx\nLDKORX2cMMzMrCbukjIzKyB3SZmZWW6cMFqI+2czjkXGscg4FvVxwjAzs5p4DMPMrIA8hmFmZrlx\nwmgh7p/NOBYZxyLjWNTHCcPMzGriMQwzswLyGIaZmeWmJRLGzp2NbkFzcP9sxrHIOBYZx6I+LZEw\ntmxpdAvMzFpf7mMYkmYAC0mS06KIWFCx/NR0+XHAORHxg7JlhwLfIXnA0i7gAxGxoWL9+MUvgiOO\nyPVjmJm1lKYbw5A0CrgamA4cA8yWNKWi2npgDnBDlU18D1gQEW8FpgLPVNvPc88NWZPNzKwPeXdJ\nTQXWRsT6iNgBLAZmlleIiA0R8Riwx6GOpLcAoyPi7rTetoh4udpOnn8+l7aPOO6fzTgWGcci41jU\nJ++EMQnYWFbelM6rxZuBLZJulPSgpAWSqh4+OWGYmeVvTM7br/YFX+ugyRjgFKCDJOn8G9AFXFtZ\nceHCLlavbgegra2Njo4OOjs7gewXRRHKnZ2dTdUel5unXNIs7WlUuTSvWdoznOWenh66u7sBaG9v\nZzByHfSWNA24PCJmpOW5QFQOfKfLrgVuLQ16S3oncEVEnJaWzwPeGRGfrlgvvvzl4NJLc/sYZmYt\np+kGvYFlwJGSDpc0FpgF3NJP/fLGLwMOlHRQWj4NeLzaSu6SSlT+miwyxyLjWGQci/rkmjAiYidw\nIbAEWAUsjojVkuZLOh1A0kmSNgIfBq6RtDJddxfwv4C7JT2SbvLb1fbjhGFmlr+WuJfUWWcFN97Y\n6JaYmY0czdglNSx8hGFmlj8njBbi/tmMY5FxLDKORX2cMMzMrCYtMYZxwAHhGxCamQ1AYccwtm71\nLc7NzPLWEgnjgAOgt7fRrWg8989mHIuMY5FxLOrTEglj4kTfsdbMLG8tMYZx0knBt74FU6c2ujVm\nZiNDYccwDjzQZ0qZmeXNCaOFuH8241hkHIuMY1EfJwwzM6tJS4xhzJ0bTJgAl13W6NaYmY0MHsMw\nM7PctETC8Gm1CffPZhyLjGORcSzq0xIJw0cYZmb5a4kxjB//OPjiF+GeexrdGjOzkaEpxzAkzZD0\nhKQnJV1SZfmpkh6UtEPSWVWWT5C0SdI3+9qHjzDMzPKXa8KQNAq4GpgOHAPMljSlotp6YA5wQx+b\n+SLQ099+nDAS7p/NOBYZxyLjWNQn7yOMqcDaiFgfETuAxcDM8goRsSEiHgNe0zcm6UTgEJJngvfJ\nCcPMLH95J4xJwMay8qZ03l5JEvB14GKg3362CRNg2zbYsWOwzWwNnZ2djW5C03AsMo5FxrGoz5ic\nt1/ti77WUfYLgNsj4tdJ7ug7aXz8412MHdvOZZfBpEltdHR07P7DKB2Cuuyyyy4XudzT00N3dzcA\n7e3tDEauZ0lJmgZcHhEz0vJcICJiQZW61wK3RsQP0vK/AKcAu4AJwD7AP0XEZRXrRURw1FFw++3w\n5jfn9nGaXk9Pz+4/lKJzLDKORcaxyAzmLKm8jzCWAUdKOhzYDMwCZvdTf3fjI+K83TOlOcCJlcmi\nnMcxzMzylft1GJJmAFeRjJcsioivSJoPLIuI2ySdBNwEtAEvA7+JiLdVbKOUMP66yvYjIpg+HT77\nWZgxI9ePY2bWEgZzhNESF+5FBLNmwcyZMLu/4xczMwOa9MK94eIuKZ9jXs6xyDgWGceiPk4YZmZW\nk5bpkvr612HzZrjyyka3yMys+blLykcYZma5ccJoIe6fzTgWGcci41jUxwnDzMxq0jJjGCtWwJw5\n8MgjjW6RmVnz8xiGjzDMzHLjhNFC3D+bcSwyjkXGsahPyySMCRPg5Zd9i3Mzs7y0zBgGwMEHw6pV\ncMghDW6UmVmTK/QYBrhbyswsT04YLcT9sxnHIuNYZByL+jhhmJlZTVpqDGP2bDjjDDj33AY3ysys\nyXkMw0cYZma5yT1hSJoh6QlJT0q6pMryUyU9KGmHpLPK5r9d0n9KWilphaSz97aviRPhueeG+hOM\nHO6fzTgWGcci41jUJ9eEIWkUcDUwHTgGmC1pSkW19cAc4IaK+VuBj6aPa30/sFDSAf3tz0cYZmb5\nyfsIYyqwNiLWR8QOYDEws7xCRGyIiMeAqJj/i4j4r/T9ZuAZ4OD+dlb0hNHZ2dnoJjQNxyLjWGQc\ni/rknTAmARvLypvSeQMiaSqwTymB9KXoCcPMLE9jct5+tRH4AZ2WJen3gO8BH+2rTldXF+3t7axb\nBytXttHT07H7l0Spz7II5fL+2WZoTyPLpXnN0p5GllesWMFFF13UNO1pZHnhwoV0dBT3+6G7uxuA\n9vZ2BiPX02olTQMuj4gZaXkuEBGxoErda4FbI+IHZfMmAD3Al8rnV6y3+7TaRx6Bj34UHn10yD/K\niNDT07P7D6XoHIuMY5FxLDKDOa0274QxGlgDvBfYDDwAzI6I1VXqXgvcFhE3puV9gDuBmyPim/3s\nY3fC2LABTj4ZNm7sq7aZmUETXocRETuBC4ElwCpgcUSsljRf0ukAkk6StBH4MHCNpJXp6mcDpwBd\nkh6W9JCk4/rbX9FPqzUzy1NLXekdAePGwe9+B2PHNrhhDeDD7YxjkXEsMo5FpumOMIabBG1tPlPK\nzCwPLXWEAXD00XDzzTCl8vJAMzPbrfBHGOBrMczM8uKE0ULKr0EoOsci41hkHIv6OGGYmVlNWm4M\n48ILk3GMT3+6gY0yM2tyHsPARxhmZnlxwmgh7p/NOBYZxyLjWNTHCcPMzGrScmMYP/whXHttci2G\nmZlV5zEMfIRhZpYXJ4wW4v7ZjGORcSwyjkV9WjJh+I61ZmZDr+XGMLZuhYMPhm3bGtgoM7Mm5zEM\nYL/94NVXYfv2RrfEzKy15J4wJM2Q9ISkJyVdUmX5qZIelLRD0lkVy+ak662RdH5t+yvuOIb7ZzOO\nRcaxyDgW9ck1YUgaBVwNTAeOAWZLqrzx+HpgDnBDxboHAl8A3gG8E5gn6XW17LeoCcPMLE95H2FM\nBdZGxPqI2AEsBmaWV4iIDRHxGFA5mDIdWBIRWyKil+QxrzNq2WlRE4afJJZxLDKORcaxqE/eCWMS\nsLGsvCmdN5h1f13rukVNGGZmeco7YVQbga/1tKxBr1vUU2vdP5txLDKORcaxqM+YnLe/CTisrDwZ\neGoA63ZWrHtPtYpdXV20t7cD0NbWxrZtHTz/fLJq6Q+kdCjqcjHKJc3SnkaWV6xY0VTtaWR5xYoV\nTdWe4Sz39PTQ3d0NsPv7cqByvQ5D0mhgDfBeYDPwADA7IlZXqXstcFtE3JiWDwSWAyeQHAktB05M\nxzPK14vKz/CFL8Do0TBv3tB/JjOzVtB012FExE7gQpIB61XA4ohYLWm+pNMBJJ0kaSPwYeAaSSvT\ndZ8HvkiSKJYC8yuTRV88hmFmNvRyvw4jIu6MiKMj4qiI+Eo6b15E3Ja+Xx4Rh0bEhIg4OCLeVrZu\nd7remyPie7Xus6gJo7I7psgci4xjkXEs6lNTwpB0fS3zmkVRE4aZWZ5qGsOQ9FBEnFBWHg2sjIi3\n5tm4WlQbw/jpT+Hzn4ef/axBjTIza3JDPoYh6VJJLwLHSXohnV4EngGa9hFFRT2t1swsT/0mjIi4\nIiImAF+LiAPSaUJEHBQRlw5TGwds4sRidkm5fzbjWGQci4xjUZ9aB71vk7Q/gKTzJH1D0uE5tqsu\nHsMwMxt6tY5hPAq8HTgO6Aa+A5wdEe/OtXU1qDaGEQHjx8OWLcmrmZntKc/rMF5Nv5VnAldHxLeA\nCQNt4HAp8i3OzczyUmvCeFHSpcBHgdvTs6T2ya9Z9StiwnD/bMaxyDgWGceiPrUmjHOA7cDHI+I3\nJHeN/VpurRoCRUwYZmZ5qvleUpLeQPIwI4AHIuKZ3Fo1ANXGMAD+9E/hU5+CM85oQKPMzJpcbmMY\nks4muXHgR4CzgaWSPjzwJg6fop5aa2aWl1q7pD4PvCMi5kTE+SRP0vv7/JpVvyJ2Sbl/NuNYZByL\njGNRn1oTxqiKLqhnB7BuQxQxYZiZ5anW6zC+RnINxr+ms84BHo2IS3JsW036GsNYuBB+9Su46qoG\nNMrMrMkNZgyj3yfuSToSeENEXCzpLOAUkken3gfcMOiWDoMDD4QHH2x0K8zMWsfeupUWAi8ARMQP\nIuJzEfFZ4KZ0WdM66KDi3YDQ/bMZxyLjWGQci/rsLWG8ISJWVs5M57XXsgNJMyQ9IelJSa/pwpI0\nVtJiSWsl3SfpsHT+GEndkh6VtErS3Fr2VzJxYvEShplZnvodw5C0NiKO6mPZLyLiyH43Lo0CniR5\npvdTwDJgVkQ8UVbnfwJvi4gLJJ0DfCgiZkmaDZwREedK2hd4HHh3RGyo2EfVMYw1a5JrMJ58sr8W\nmpkVUx7XYSyX9BdVdvQJoJYRgqnA2ohYHxE7gMUk96MqNxO4Ln3/feC09H0A+6e3IdmP5ErzF2rY\nJ5B0ST37bK21zcxsb/aWMC4CPiapR9KV6fQT4M+Bz9Sw/UnAxrLypnRe1ToRsRPYImkiSfLYBmwG\n1gFfj4jeGvYJQFtbcrfaXbtqXWPkc/9sxrHIOBYZx6I+/Z4lFRFPA++S9B7g2HT27RFxd43br3a4\nU9l/VFlHaZ2pwKvAG4GDgJ9J+lFErKtlx2PGwIQJ0NubjGeYmVl9+k0YJRFxD3DPILa/CTisrDyZ\nZCyj3EbgUOCptPvpgIh4XtK5wJ0RsQv4raR7gZNIjjb20NXVRXt7OwBtbW10dHTQ2dnJQQfBf/xH\nD5MnQ2dnJ5D9wmjFcmdnZ1O1x+XmKZc0S3saVS7Na5b2DGe5p6eH7u5ugN3flwNV880HB7XxJAGs\nIRn03kxyP6rZEbG6rM4FwLHpoPcs4Mx00PtvgaMj4hPp0/4eAM6JiMcq9lF10Btg6lT45jdh2rRc\nPp6Z2YiV5wOUBiUdk7gQWAKsAhZHxGpJ8yWdnlZbBLxe0lqSMZPS6bPfAiZIegxYCiyqTBZ7U7Rr\nMSp/TRaZY5FxLDKORX1q6pKqR0TcCRxdMW9e2fvtJHfArVxva7X5A+EzpczMhk6uXVLDob8uqc98\nBt70JrjoomFulJlZk2u6LqlG89XeZmZDp6UTRtG6pNw/m3EsMo5FxrGojxOGmZnVpKXHMO66C668\nEpYsGeZGmZk1OY9hVPARhpnZ0HHCaCHun804FhnHIuNY1KelE4bPkjIzGzotPYYRAWPHwtatyauZ\nmSU8hlFB8lGGmdlQaemEAUnCKMo4hvtnM45FxrHIOBb1afmEUbQbEJqZ5aWlxzAAZs6Ej30Mzjxz\nGBtlZtbkPIZRRdFOrTUzy0vLJ4wiDXq7fzbjWGQci4xjUZ+WTxg+wjAzGxq5j2FImgEsJElOiyJi\nQcXyscD3gBOB/yZ5DOuGdNlxwDXAAcBO4B0R8UrF+v2OYfzzP8OyZfDtbw/dZzIzG+mabgxD0ijg\namA6cAwwW9KUimqfAJ6LiKNIEstX03VHA9cDn4yIY4FOYMdA21CkLikzszzl3SU1FVgbEesjYgew\nGJhZUWcmcF36/vvAaen79wGPlJ7jHRHP93so0YcidUm5fzbjWGQci4xjUZ+8E8YkYGNZeVM6r2qd\niNgJbJE0EXgzgKQ7JS2XdPFgGlCkhGFmlqcxOW+/Wv9Y5VFCZR2ldcYAJwMnAS8DP5a0PCLuqdxg\nV1cX7e3tALS1tdHR0UFnZycAa9b08NRTkPRoZb8wSstbqdzZ2dlU7XG5ecolzdKeRpVL85qlPcNZ\n7unpobu7G2D39+VA5TroLWkacHlEzEjLc4EoH/iWdEdaZ2k6brE5Ig6RdA4wPSI+ntb7O+CliLiy\nYh/99lS99BIceGDyqgEN75iZta6mG/QGlgFHSjo8PRtqFnBLRZ1bgTnp+48Ad6fv7wKOkzRe0hjg\n3cDjA23AvvvCqFGwbdug2j+iVP6aLDLHIuNYZByL+uTaJRUROyVdCCwhO612taT5wLKIuA1YBFwv\naS3wLElSISJ6JX0DWA7sAm6PiDsG047SOMb++w/BhzIzK6iWv5cUwNvfDtddBx0dw9QoM7Mm14xd\nUk3BZ0qZmdXPCaOFuH8241hkHIuMY1GfQiQMX+1tZla/QoxhXHZZMuD9+c8PU6PMzJqcxzD6UJQu\nKTOzPBUiYRTlud7un804FhnHIuNY1KcQCcPP9TYzq18hxjDuvRcuvhj+8z+HqVFmZk3OYxh9KEqX\nlJlZngqRMIrSJeX+2YxjkXEsMo5FfQqRMCZOhN5e2LWr0S0xMxu5CjGGAdDWBr/6VXKrczOzovMY\nRj98tbeZWX0KkzCKcPGe+2czjkXGscg4FvVxwjAzs5oUZgzj3HPhAx+A884bhkaZmTW5phzDkDRD\n0hOSnpR0SZXlYyUtlrRW0n2SDqtYfpikFyV9rp52FOXUWjOzvOSaMCSNAq4GpgPHALMlTamo9gng\nuYg4ClgIfLVi+TeA/6i3LUXoknL/bMaxyDgWGceiPnkfYUwF1kbE+ojYASwGZlbUmQlcl77/PvDe\n0gJJM4H/AlbV2xBf7W1mVp+8E8YkYGNZeVM6r2qdiNgJ9EqaKGk/4G+B+cCA+tmqKUKXVGdnZ6Ob\n0DQci4xjkXEs6jMm5+1X+6KvHKGurKO0znzgHyNim6S+tgVAV1cX7e3tALS1tdHR0bH7D6N0CHrQ\nQZ08+2xWrlzusssuu9zK5Z6eHrq7uwF2f18OVK5nSUmaBlweETPS8lwgImJBWZ070jpLJY0GNkfE\nIZJ+CkxOqx0I7AS+EBH/VLGPms6SeuABuOACWL58SD5aU+rp6dn9h1J0jkXGscg4FpnBnCWV9xHG\nMuBISYcDm4FZwOyKOrcCc4ClwEeAuwEi4o9KFSTNA16sTBYDUYQuKTOzPOV+HYakGcBVJOMliyLi\nK5LmA8si4jZJ44DrgeOBZ4FZEbGuYhulhPGNKtuv6QijtxcOPxy2bKn7I5mZjXiDOcIozIV7EbDP\nPvDSS8mrmVmRNeWFe81CSu5U28rdUqUBLnMsyjkWGceiPoVJGOBxDDOzehSmSwrg5JNhwQI45ZSc\nG2Vm1uTcJbUXvtrbzGzwCpUwWr1Lyv2zGcci41hkHIv6FC5h+AjDzGxwCjWG8aUvwe9+B1dckXOj\nzMyanMcw9qLVu6TMzPJUuITRyl1S7p/NOBYZxyLjWNSnUAnDZ0mZmQ1eocYwVqyA88+HRx/NuVFm\nZk3OYxh74TEMM7PBK1zCaOUuKffPZhyLjGORcSzqU6iEse++yV1rt21rdEvMzEaeQo1hAEyaBEuX\nwuTJe69rZtaqPIZRg1bvljIzy0vuCUPSDElPSHpS0iVVlo+VtFjSWkn3STosnf/HkpZLekTSMknv\nGYr2tPKpte6fzTgWGcci41jUJ9eEIWkUcDUwHTgGmC1pSkW1TwDPRcRRwELgq+n83wKnR8TbgS6S\nx7jWzWdKmZkNTq5jGJKmAfMi4v1peS4QEbGgrM6daZ2lkkYDv4mIg6ts67fA70fEjor5AxrD+OQn\n4cQT4S//cnCfycysFTTjGMYkYGNZeVM6r2qdiNgJ9EqaWF5B0oeBhyuTxWC0cpeUmVmexuS8/WrZ\nq/JwoLKOyutIOga4AviTvnbS1dVFe3s7AG1tbXR0dNDZ2QlkfZalcm9vD7/6FUD15SO5XN4/2wzt\naWS5NK9Z2tPI8ooVK7jooouapj2NLC9cuLDf74dWLvf09NDd3Q2w+/tyoIajS+ryiJiRlqt1Sd2R\n1il1SW2OiEPSZZOBHwNzIuL+PvYxoC6p734XfvYzuPbaQX+sptXT07P7D6XoHIuMY5FxLDKD6ZLK\nO2GMBtYA7wU2Aw8AsyNidVmdC4BjI+ICSbOAMyNilqQ2oAeYHxE39bOPASWMm2+GRYvgllsG9ZHM\nzFpC041hpGMSFwJLgFXA4ohYLWm+pNPTaouA10taC1wEzE3n/xVwBPD3kh6W9JCk19fbJo9hmJkN\nTuGu9H78cfizP4PVq/ded6Tx4XbGscg4FhnHItN0RxjNyFd6m5kNTuGOMF55BfbfP3nVgHKrmVnr\n8BFGDcaOTe5a+8ILjW6JmdnIUriEAa3bLVV+DULRORYZxyLjWNSnkAnDZ0qZmQ1c4cYwAN73Pvib\nv4Hp03NqlJlZk/MYRo1OPBE+/Wn43vfg1Vcb3Rozs5GhkAnjy1+Ga65JrvieMiW5TciOum9r2Hju\nn804FhnHIuNY1KeQCUOC006Dn/wkSRr/8i9w9NHwne8kp9uamdlrFXIMo5qf/xz+4R9g1So44QT4\ngz/Yc3rTm2C//YagwWZmTaDpbj44HIYqYZSsWZPcNuSXv9xzWrcuuX5j7FgYMyaZ9tlnz/fVprFj\nYfz4ZNp33z2n/faDtra+p1GFPP4zs+HghJGjXbvg+eeTQfIdO5LX0vtSufS+fHrlFXj5ZXjppey1\nNG3dClu2QG9v9trbm+xn61b4vd+DQw9NpsmTs/fHHgtHHPHahOL75GQci4xjkXEsMoNJGHk/QKll\njBqVXPA3XLZvh1//GjZuTKZNm5KjnyVLYOXKJLEcf3xyxtcJJyTTzp3D1z4zKx4fYYxQv/0tPPww\nPPggPPRQMj33HLz//XDmmcnrhAmNbqWZNSt3SRXc5s1w661w001w771w6qlJ8jjjDHjjGxvdOjNr\nJk154Z6kGZKekPSkpEuqLB8rabGktZLuk3RY2bJL0/mrJb0v77aOdGvW9PDJT8IddyRdWOefD3ff\nDW95C5x0UnKx4g03JIP4rZ5jfb59xrHIOBb1yXUMQ9Io4GqSR7Q+BSyTdHNEPFFW7RPAcxFxlKRz\ngK8CsyS9FTgbeAswGfiRpKN8ONG3FStW7B7QO+AAOOecZNq+HZYvh/vugx/8AC6+OBnvmDYtmSZN\nys7iKj+ba/z45Fbw++2XvO6778g5c6s8FkXnWGQci/rkPeg9FVgbEesBJC0GZgLlCWMmMC99/33g\nf6fvP0jySNdXgXXpI1ynAktzbvOI1dvbW3X+uHFw8snJBMnRxcaNcP/9sHRp8hTCyjO5Xn4Ztm3L\nzuYqvR83Lkke48bt+TyRvt5XlkePTpJOtdfSacqVr+PGZacmV04TJiTJsTS97nXJ64YNvTz7bFZv\n9OghCvII1NffRRE5FvXJO2FMAjaWlTeRfOlXrRMROyVtkTQxnX9fWb1fp/OsThIcdlgynX127evt\n2pUkkq1bk6OWkvJjvsrjv8plu3YlRzeVrzt37nnKcvlr6dTkyumll+C//zt5tkn5tGVLcobZdddl\n9UaPzo6e9tknSVLVpv4eqjWYB24Ndnv17qv8/dNPJ+NafW1byqbKcikm5e9HjUquLxo37rVT6Tql\nymuUxoxJllWuVyqPH5/NK72v/HGwzz4Dj4kNrbwTRrU/+8oupb7q1LKulVm3bl2u2x81KumeGglX\nvHd1raO7O3kfkSSeUqJ55ZUseZVP/Z2WPJiO0P7WGeyyWtapXP/v/m4dX/xi9WUR2VRZLk27du35\nfteuJIbbtydT5fvya5RefTWJeSnx97Xe9u1JvcrX8h8H0p5dp2PH9n3BbKnrtDIRPvbYOu67LytX\ney3p76i5VsP1VM/PfQ7e857895PrWVKSpgGXR8SMtDwXiIhYUFbnjrTOUkmjgc0RcUhlXUl3AvMi\nYmnFPpxEzMwGodku3FsGHCnpcGAzMAuYXVHnVmAOydjER4C70/m3ADdI+keSrqgjgQcqdzDQD2xm\nZoOTa8JIxyQuBJaQnMK7KCJWS5oPLIuI24BFwPXpoPazJEmFiHhc0r8BjwM7gAt8hpSZWeOM+Av3\nzMxseIyQs+qr29tFga1M0iJJT0t6tGzegZKWSFoj6S5Jr2tkG4eLpMmS7pb0uKSVkv46nV+4eEga\nJ2mppIfTWMxL57dLuj+Nxb9KKsR95CSNkvSQpFvSciHjACBpnaRH0r+NB9J5A/ofGbEJo+yiwOnA\nMcBsSVMa26phdS3JZy83F/hRRBxNMhZ06bC3qjFeBT4XEW8F/hD4q/RvoXDxiIjtwHsi4nigA3i/\npHcCC4Ar01j0klwwWwSfIenWLilqHAB2AZ0RcXxElC5vGND/yIhNGJRdFBgRO4DSRYGFEBE/B56v\nmD0TuC59fx1w5rA2qkEi4jcRsSJ9/ztgNcndAYoaj23p23Ek45QBvAe4MZ1/HfChBjRtWEmaDHwA\n+E7Z7NPpvbn4AAAEaElEQVQoWBzKiNd+5w/of2QkJ4xqFwUW/cK+QyLiaUi+RIGDG9yeYSepneSX\n9f3AG4oYj7Qb5mHgN8D/A/4L6I2IXWmVTcDvN6p9w+gfgYtJr9+SdBDwfAHjUBLAXZKWSfrzdN6A\n/kdGcv+dL+yzPUj6HyS3l/lMRPyuqNfopF+Ix0s6ALiJ5H5sr6k2vK0aXpL+FHg6IlZI6izN5rXf\nGy0dhwrviojfSDoYWCJpDQP8/CP5CGMTcFhZeTLJDQ6L7GlJbwCQ9EbgmQa3Z9ikg5ffB66PiJvT\n2YWNB0BEvAD8BJgGtKXjflCM/5WTgQ9K+iXwryRdUQuB1xUsDrulRxBExG+BH5J06w/of2QkJ4zd\nFwVKGkty/cYtDW7TcKv8xXQL0JW+nwPcXLlCC/su8HhEXFU2r3DxkPT60pkukvYF/phk0Pcekgtj\noQCxiIjLIuKwiPgDku+GuyPiPAoWhxJJ+6VH4EjaH3gfsJIB/o+M6OswJM0AriK7KPArDW7SsJH0\nf4BO4CDgaZI7/v4Q+HfgUGAD8JGIaPnbc0o6GfgpyT9ApNNlJHcG+DcKFA9JbyMZvByVTv83Ir4k\n6U0kJ4YcCDwMnJeeLNLyJL0b+JuI+GBR45B+7ptI/jfGADdExFfSG73W/D8yohOGmZkNn5HcJWVm\nZsPICcPMzGrihGFmZjVxwjAzs5o4YZiZWU2cMMzMrCZOGFZ4kl5MXw+XVPlEyHq3fWlF+edDuX2z\n4eSEYZbdT+dNwLkDWbHsNhN9uWyPHUWcMpDtmzUTJwyzzBXAKekDdz6T3vX1q+kDiVZI+gtIrhyW\n9FNJN5M+a0HSTeldQFeW7gQq6Qpg33R716fzXiztTNLX0vqPSDq7bNv3SPp3SatL66XLviJpVdqW\nrw5bVMxSI/lutWZDbS7pLSQA0gTRGxHvTO9Xdq+kJWnd44FjImJDWv5YRPRKGg8sk3RjRFwq6a8i\n4oSyfZRutf1nwHER8TZJh6Tr/CSt0wG8leT25PdKehfJMz7OjIgp6foH5BUEs774CMOsb+8Dzk+f\nLbEUmAgclS57oCxZAFwkaQXJczgml9Xry8kkd1ElIp4BeoB3lG17cyT37VkBtAMvAC9J+rakDwEv\n1fnZzAbMCcOsbwI+nT7S8viIOCIifpQu27q7UnJzu9OAd0ZEB8mX/PiybfS17b7K28ve7wTGRMRO\nkttR3wicDtw5mA9kVg8nDLPsy/pFYELZ/LuAC9JnbSDpKEn7VVn/dSRPctuePkt8WtmyV0rrV+zr\np8A56TjJwcCpJHfXrd7AZL9tEXEn8DnguNo/ntnQ8BiGWXaW1KPAzrQLqjsirkof+fqQJJE8XKba\nM4/vBD4laRWwBrivbNk/A49KejAiPlraV0TcJGka8AiwC7g4Ip6RVPl0vFLbDgBuTsdIAD47+I9r\nNji+vbmZmdXEXVJmZlYTJwwzM6uJE4aZmdXECcPMzGrihGFmZjVxwjAzs5o4YZiZWU2cMMzMrCb/\nH1aUyLeQLgphAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fea57844c10>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(T.J)\n",
    "plt.grid(1)\n",
    "plt.xlabel('Iterations')\n",
    "plt.ylabel('Cost')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#Test network for various combinations of sleep/study:\n",
    "hoursSleep = np.linspace(0, 10, 100)\n",
    "hoursStudy = np.linspace(0, 5, 100)\n",
    "\n",
    "#Normalize data (same way training data way normalized)\n",
    "hoursSleepNorm = hoursSleep/10.\n",
    "hoursStudyNorm = hoursStudy/5.\n",
    "\n",
    "#Create 2-d versions of input for plotting\n",
    "a, b  = np.meshgrid(hoursSleepNorm, hoursStudyNorm)\n",
    "\n",
    "#Join into a single input matrix:\n",
    "allInputs = np.zeros((a.size, 2))\n",
    "allInputs[:, 0] = a.ravel()\n",
    "allInputs[:, 1] = b.ravel()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "allOutputs = NN.forward(allInputs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEPCAYAAABBUX+lAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXd8lNeZ77/PNHWhCgLUG724gg02MmBjmxjbOMVxHJfk\nZu/N7qZs4r3JdTaxneRmk002uVnHaRsntlOc5m7HxgVwwXHvNh2BQIBACJBQH825f7wjkLFAbc57\n5p05Xz7z0WiYec+DgN8885zn/B5RSmGxWCyWxMdnOgCLxWKxuIMVfIvFYkkSrOBbLBZLkmAF32Kx\nWJIEK/gWi8WSJFjBt1gsliQhoHsBEdkOHAYiQK9S6kzda1osFovlg2gXfByhr1NKHXRhLYvFYrGc\nADdKOuLSOhaLxWI5CW4IsQJWicjLIvIZF9azWCwWyyC4UdI5Wym1V0QKgSdEZL1S6jkX1rVYLBbL\nALQLvlJqb/TrfhG5DzgTeJ/gi4g19LFYLJYRopSSkTxfa0lHRNJFJDN6PwO4AHhnsOfu3a9QSs+t\nJazIWq+IRPStoZSiSe3lx+qHY7rGTTfdNLrXNpWierdq/fON+RaJoC7PQrW26Ps5JOBtqJ/F/ddf\nz0s//anxOOPhZ5FMt9Ggu4Y/AXhORF4HXgAeUko9PtgT6xv1BZHrBz9woE/fGgA55HKIQ0SI6F1o\nMPzV0LfF/XVHgghMngK7NpqOJKEoOftsdj3/vOkwLB5Aq+ArpeqVUnOVUqcopWYppb57oufW79IZ\nCVSGoL5X7xohQqSRRhttehcajEA1hONc8AGKp8DODaajSChKFiyg4Tm7LWYZmrhpl9Qt+BVB2Naj\ndw2AXPJooWXUr6+rqxvdC/3V0Ld51Ou6RslU2DW04I/655CADPWzKJg6le62NlobNX5MjhPsv4ux\nET+Cr/nfqhsZPkAeeRw0IfiBaghvHfW6rlE8vJKO/Y99jKF+FiJCaZJk+fbfxdiIG8HftlPv9Std\ny/Bzx5Thjxp/DYQ9kuHbkk7MKT3nHBqefdZ0GJY4J24EX3eGX+FShp87xgx/1AQqoW87KM0702Nl\ncg3srYewC38ZSUTpwoVJkeFbxkbcCH5jE/Rq1IDKIGx1IcPPG2MNf9RIOvjyoU/zR6WxEkqFgsmw\nZ5vpSBKKiaeeSsuWLXQdOmQ6FEscEzeCX1QAu5r0Xb8sBI1hCGs+4mUswwcI1ECfV+r4tqwTS/yh\nEJPPOINdL7xgOhRLHBM3gl9ZoreOHxKY4IcGzZWETDIJE6aLLr0LDYa/2iN1/GnQsN50FAlHycKF\n7LB1fMtJiBvBr5isv45fFYJ6zWUdQcgl11Ad3wOHrwBKp8NOK/ixpnThQnauW2c6DEscEzeCrzvD\nB6c1c5tLG7dmOnU8cviq1Gb4Oig56yx2v/IK4e5u06FY4pT4Efxi2Kb7tK1LrZnGNm69kuGXTHMy\n/FH6gVgGJyU7m/zaWva8+qrpUCxxStwIfkWxO/YKbmX4Rko6/ioIbwNlwMtnJGTlQmoGNGv+C09C\nShcupMGWdSwnIG4Ev7IYtrpw+CqhWzN9meAbBxEPHLEvnW7LOhooXbiQnbYf33IC4kbwx+dDZze0\nHtG3hlfsFcaEV+r4/WUdS0wpWbCAhnXrUJE4/5RnMULcCL5ItI6vMcsv8EOvgkOaD6OOI4dWWunD\nwKnXQI03TNTsxq0WsidPJiU7m+YN9pyD5YPEjeCDI/g6WzNFnLKO7tbMAAGyyOYQBk49esUm2Wb4\n2ig791zbj28ZlPgS/BLY2qB5jRBsdamsY03UToLN8LVRdu657Hj6adNhWOKQ+BJ8zRk+uNuaeZAD\n+hc6nkCNN1oz8yZCbze0GvgZJThl557LjmeeGfUYPEviEleCX6G5hg/RDN8Dg1BGjT/qix/vrZki\nNsvXRG5VFSoS4VB9velQLHFGXAm+K62ZrpZ0DGSvvkzw5ULEAz3uto6vBRGh7Nxz2W7LOpbjiCvB\nL58MDXtAZ0eZG5u2APnkm8nwwSnr2Dp+UlN27rl2IIrlA8SV4KenQd44xxtfF+Uh2BV22jN14py2\nPUgEA6UVfw2EN7m/7kixGb427MatZTDiSvDB6dTRuXEbEigKwE7NZZ2U6K8jtOldaDBsL37SUzh9\nOl2HDiXFYHPL8Ik/wS/W35pZ5ZLFQi55HDBiolbrjZLOhAo41ARd7aYjSTjE56P0nHPY8cwzpkOx\nxBFxKfjaWzNdMlHLJ99Ma6ZXevH9fphUDTs3mo4kIbF1fMvxxJ/gu3D4qsrF1kwzGX5VdKB52P21\nR4qt42ujvx/fYukn/gQ/gQ5fORm+iYHmqeCf4Ih+vGPr+NoomjuXww0NdBywh9ssDvEn+C5Mvqpy\n0Rf/gImSDoDfI3V8O+5QG75AgJKzzqLB2iVbosSd4E8shNZ2OKJxH6+/pKP75LnTi38AhYEj7l7p\n1CmbATveNR1FwlJq2zMtA4g7wRfRP9A81+/8wVs0uxenkYYgdNChd6HB8MrG7aQaaNoOPXYOqw7K\n6+qs4FuOEneCDy5u3Gou6whizmIh4JHDV6EUmFAOjR6I1YNMPuMMDmzaROfBg6ZDscQB8Sn4bgw0\nd6lTJ8+UxUKg1hslHXDKOg3vmY4iIfGHQhTPn2/r+BYgXgW/RL+JmluHr4xl+P4K6GsE5YFSSel0\nW8fXSFldHdvXrDEdhiUOiEvBr3KpUyehM3wJgr8U+jxgkVs2w7ZmasTW8S39xKXgu2GT7EYNHwwK\nPkQtFjxQGy+dDg02w9eFreNb+olLwa8odmyS+zR20VSF3Jx8ZUjwvdKpU1wLe7ZBrwt/IUmIPxRi\n8rx5to5viU/BT0uF/BzYtVffGpMD0NwHnZrdi7PIojv6y3UC1d7YuA2lwvhS2O2B0YwexZZ1LOCS\n4IuIT0ReE5EHh/uaKs0bt36BMhcsFgSJjjs01ZrpAcEHu3GrmfK6OravXWs6DIth3MrwvwCMqO+u\nskR/a6Zbdfx8Y/NtPWKvALY1UzOTzjiDAxs30nX4sOlQLAbRLvgiUgxcDPxqJK+rcuPwlWu++Plm\nPHX8pRDZD8rASd+RYjN8rQRSUph85pnWLjnJcSPD/xHwrzAyQ5kqtzJ8V1wzDZV0xA+BSgh7oDZe\nNsOaqGmmbNEiO9g8yQnovLiILAealFJviEgdICd67s0333z0fl1dHZUlda60Zq46oncNcFoz3+Zt\n/QsNhr8WwhshONvM+sOleIqzadsXBr/Wf5ZJS3ldHY/fcIPpMCyjZO3atawd4z6M7v9ZC4AVInIx\nkAZkichdSqlrjn/iQMEH2N+i//BVtau9+IZskgNTvNGLn5IG+ZNh91YomWI6moRk8rx5NK9fT9fh\nw6SOG2c6HMsIqauro66u7uj3t9xyy4ivobWko5S6USlVqpSqBK4EVg8m9oNRkAvhPjiocY+pIggN\nvdCn2b14HOPooIMeDPSZB2qhzyMjBG0dXyuBlBQmz5tnp2AlMXHZhw+OTbJuE7VUHxT6YafmLN+H\njxxyOYiBk45eyfAByuwwFN1ULF5sfXWSGNcEXyn1tFJqxUheU1WaOPNt88jjAM36Fzqe/hq+7mkv\nscBm+NopP+886levNh2GxRBxm+GDe546W1zpxS8w05rpK3C+Rgy82YwUK/jamXT66Ryqr6ej2QP/\nHiwxJ64F35XWTJd68c21Zkq0ju+Bsk7JVNi9Wa+JUpLjDwYpWbDAtmcmKfEt+KUuuWa6ZJNsbKB5\nIFrWiXfSMiG3CPZsNR1JQlN+3nm2jp+kxLfgu3Dattq1w1f5Bl0zp3jLYsGWdbRiN26Tl7gW/NKJ\nsLcZujUKclUItvTo39PMZhzttNOLCxsGx+OVDB+gbCZsf8d0FAlN0dy5tO3Zw5G9Gu1oLXFJXAt+\nIADFRbC9Ud8aOX6nPXOf5rKxHz/jyDGT5QemeKOGD1ETNZvh68Tn91N2zjnU2yw/6YhrwYfEMlHL\nN2aiVg3hraA8sBlaNsNm+C5g6/jJiTcE3w2LhUTeuPVlOO2ZfZp/kLGgdJqzaRs2UPpKIqzgJydx\nL/jVpe64Zm5xKcM356njkdbMUCoUlkCjRzaZPcqEWbPoPHiQ1l2a/3NZ4oq4F/yqEtjiQqeOG4Jv\nWzOHSdlM2GHLOjoRn4/yujpbx08y4l/w3bJXSOTJVxC1WPBAhg+2NdMlKpYsof6pp0yHYXGRuBf8\nymKob4SIxmHjbmX448jhCG2GWjOneCfDL7etmW5QuWQJ2558EuUFnyVLTIh7wc9Ih5ws2L1P3xrj\n/dCt4JALrZk55BhyzfRIDR9shu8SeTU1iM/HgU0e+XdhGTNxL/ig32JBxGnNdK+Ob8I1sxz69oLq\ndH/tkTK5FvbtgJ4u05EkNCJCpS3rJBXeEPyEslgoMFPHlwD4K5x+/HgnGIKiCtjlkRKUh7F1/OTC\nE4JfXZo4nTrGXDPBg506tqyjm4rFi6lfs4aIdShNCjwh+G4cvnKrUyfPlC8+2Dq+5QNkTZpEZlER\ne19/3XQoFhewgh8lOTJ8D3XqWMF3jYolS+wUrCTBG4IfLeno7B5z67St05p5xFBr5lQIb3B/3dFg\nBd817MZt8uAJwc/Pcb62HNa3RnEADvZBu8Z+fxjommmgNdM/xTvzbSdVQ/Mu6PZAV5HHKa+rY+fz\nzxPu7jYdikUznhB8kejG7Q59a/gEKlx1zTTQmukrAAQi+91fe6QEgo7oN6w3HUnCk5qTQ8HUqex6\n4QXToVg04wnBh8RyzTRmkywS9cb3Sh3fduq4ha3jJwdW8AdQHYLNiZzhg7fq+OUzYfvbpqNICiqW\nLKH+ySdNh2HRjGcEv9qFgebu+eKbbM30UKdOuXXNdIvShQtpeustug5r3CizGMczgl/l0uErtzJ8\nc62ZUz0k+LOg3mb4bhBMS6N4/ny2r11rOhSLRrwj+G7ZK7jQLTmOcXTQQQ8uvLscj99DGf6Ecmg/\nBEcOmY4kKag8/3y22bJOQuMZwZ80Hg61QXuHvjVKg7A3DF2aWzN9+Mgl14ynTqAK+hpAGXizGSk+\nH5ROt1bJLlG5dCnbnnjCdBgWjXhG8H0+qJist44fECgLwjZXhqEUmNm4lRD4S7xhogZOp47duHWF\norlz6TxwgMM7PTD72DIqPCP4ANVlibNxa6w1E7zVmlkxy2b4LiE+HxXRoSiWxMRbgq/58BUky8at\nh+r4NsN3FVvWSWw8J/i6M/wa25oZX1TMcg5fecEOIgGoXLqU+qeeQumcKWoxhqcEv6rEhdZMlyZf\nFZg8fOWf4p3DVznjnQ2clj2mI0kKcsrLScnOpult+6kqEfGU4LuR4bvlmplFNt1004WBMX5eyvBF\nomUdW8d3i4polm9JPIYUfBG5R0SWi4jxN4fSic4w826Nglwegl1h6NFcQRCEPFN1fN8EIAwRQ58w\nRkq53bh1E1vHT1yGI+I/A64CNovId0VkquaYTkgwCCVFsL1R3xohgckB2JHInTr9JmpeyfKtxYKr\nVJx3Hg3r1tHX44GzGpYRMaTgK6WeVEp9AjgV2A48ISLPi8j1IhI82WtFJEVEXhSR10XkbRG5aawB\nV5fpr+PXJIWJmscE32b4rpGWl0fB1Kk0rFtnOhRLjBlWmUZE8oHrgP8BvA78GOcN4KSf+5RS3cB5\nSqlTgLnARSJy5lgCdmuguXuCb6hTx0sWC2UzoeE9sJ0jrlG1bBlbV60yHYYlxgynhn8v8CyQDlyi\nlFqhlPqTUupzQOZQr1dK9ZshpAABYEzVcbc8ddzYuDXbmukhE7WMbMjOh6btpiNJGqouuMDW8ROQ\nwDCe8xOl1KCTEZRSpw/14uhm76tAFXCbUurlkYX4fqpL4XHNnzRXZsFBF5JJoyWd0DngyzWz9mgo\nm+EcwJpYaTqSpKB4/nxatm6lfd8+MsaPNx2OJUacMMMXkZUishLI6b8/8DbcBZRSkWhJpxiYJyLT\nxxKwG62ZZSGYm6p3DYBMMokQoRMDc1v9EyBlifvrjhY7/cpV/MEg5XV11mYhwThZhn9J9Ot44Gyg\nP8s/D3geuHckCymlWkVkLXAh8N7xv3/zzTcfvV9XV0ddXd2g16kohh27IRyGwHA+n8Qx/a2ZB2im\nmBLT4cQ35TPh5UdNR5FUVF1wAVtXrWLWVVeZDsUCrF27lrVjnFcgaogj6yLyBHCNUmpP9PuJwB1K\nqWVDXlykAOhVSh0WkTRgFfBdpdTfjnueGiqOgZQtgTV3QGUCaORf+BO1TGEOc02HEt9seR2+/0n4\nhe3WcYuWLVv4zbnn8qXGRkTEdDiW4xARlFIj+osZTpdOcb/YR2kCSod5/YnAGhF5A3gRWHW82I+G\n6lLYrNlEzS3yyKfZVB3fS5ROgz1boafbdCRJQ25VFYHUVPa/a0tpicJwiiJPicgq4G6cDpsrgWEV\n9pRSb+O0b8YUN+r4blFAAZvwSLeMSUKpUFQBuzZA5RzT0SQFIkLl+eez9YknGD9zpulwLDFgOAev\n/hn4BTAHp5f+l9GWTGMkUoafbzP84WNn3LpO1QUXsO3xx02HYYkRwzp4pZS6Vyn1L9HbfbqDGgo3\nBqHA4I68rX2wPoZVhf7JV2psxxNiT8fvQPWZjuL9VMyG+rdMR5FUVCxeTMO6dYS7bSktERjOwas2\nEWmN3rpEpE9EWt0I7kRUl+o/fLW2HZqjetcRgU/sgorNcGY9zNgK22J0MCuNNIIEOUJbbC44WlTE\neYdT0QMI7d+PPwvlitl2GIrLpOXmUjh9OjutzUJCMJySTpZSKjt6SwWuAH6qP7QTU1UC23bpPWn/\nb/vgzahzcboPDkXg5kLYUA0fyY7tSdx8CsyUdXpfg857oPddEJ9jqtb/SSN0FvRtdj+mk1E+y2b4\nBqi64AK2WJuFhGDElsdKqfuBIVsydZKeBnnjoLFJ3xpTUhyR76cqBJ3R7ycE4I0Y2tgb89Tp/Ct0\n/Be0/xAOLIKWC6HjNghvg0gL9L7hfkwnY0IZdLZBW4vpSJKK6gsvtL46CcKQXTrHnar1AaeDiakd\n76d/47Zkop7rz06B+9sg0+dk8+u74cNZzu+dlw5N4ditZSzDz/6OU6fveRoiB4Aw9DwPnddAZBf4\nh9t96xIixzZuZy8yHU3SMPnMMznc0EDb7t1kTZpkOhzLGBhOW+YlA+6HcSySL9USzQjod81cPF/P\n9T8+DroUPNQG5UH4diHMS4ewguVZjm9+RIEvBudR8slnF5o3JU6E+CFl8bHv0z4Oqhv6dgBxeNim\nfJZTx7eC7xq+QIDKpUvZ+vjjzL3uOtPhWMbAcAT/V0qp9+3YiMgCYJ+ekIaH7tbM8QH4SgH8vAV+\ndxj+0ArNYcjzw0WZcHmW8wYQCwoooNmUayZA1yPQeZdT01eHwDceQudC6uUQqDEX12CUz4L6N01H\nkXRULVvGlsces4LvcYZTw791mI+5ihuHr548Aq93wRXZ8L3xsK4CniiDaSnw6T3QG6NOyjzyOcRB\nIhjwe+95Frruh+A8yPk9FG6Dgjcg9cNw+HPQp3G82GiosL34JqhetoxtTz5JpC/OWnUtI+KEGb6I\nnIVjmlYoIl8a8FvZgF93YEPhxiCUG/fB1wvhkqxjj0UUXJsDPzwA23ugJmXs6wQJkkEmhzhEHnlj\nv+BIaLsF0q6F9E8ee0wpx0nTNx7Cm8A/2d2YTkb5LGh412nR8hkfs5w0ZBcXk1lUxO5XXqF43jzT\n4VhGycn+x4RwBpwEgKwBt1bgw/pDOzlV0Qx/BJ5rIybNB+3HJd29yinzzE2FlBjqjXPidn/sLjhc\nfIUQOS6LVy3Q/l8QnAX+MvdjOhlZuZCeDfsS5Ki1h7BTsLzPCTN8pdTTwNMicodSageAiOQCh0Zk\nbamJ7EzITIc9+2GSpvkM/5ALa9ph1RHY1gvbex3BX5gO/7cQSk860Xdk5JNPi4k6fvqnoesBOHiV\n03ff1wD4ILQIMr8OgTgcONLfqVNUYTqSpKJ62TKevuUWFn3jG6ZDsYySk5V0vgH8WSm1QURSgEdx\nvHTCInKVUsr4ZIT+jVtdgv+JcbA4HVZ3QIY4vfnVIQhGm1eUip5VigHGWjNTlkLwLKc1U9Kd0Yf+\nomO/H8s/ZKzo79Q5a4XpSJKK0nPOoemtt+g6dIjUnBzT4VhGwcmKEh+DozaO10afWwgsAr6jOa5h\nUVOm12JBKadb5xPj4LJsZ7M2KFEHghjrYIHJ+bYSgNSLIaXumNirvvgUezgm+BZXCaalUbJgAfWr\nB514avEAJxP8ngGlm2XA3UqpPqXUeobXzqkd3a2ZrRG445Bzv0c5G7bgaGCsdbAgaqJmhI7/jr6L\n9R3bFBG/84c0X737ILZTxxhVy5ZZmwUPczLB7xaRmSJSiDPWcKBHaow60MdGTZneTp0MnyP64By0\n6j9kFdGggePI4QhH6KU39hcfkgCotmMiD6Cic3bjMcMvmQp7t9lhKAaoXraMrY89Rhxs41lGwcky\n9S8Af8Up4/xIKVUPICIXA6+7ENuQ6G7NDAj8S75zf0M3vNYFB/ogCJQFYWoKVIRis5YfPznkcJAW\nxjMhNhcdDkpBxv9y7vc1QPdaiOwDIiAZEKh1NnAlRn/QWBBKhQnldhiKAQqmTQOgef16CqdPNxyN\nZaScrEvnRWDqII//DRjzmMJY0F/S0VlqPhKBn7U4ZmmTgo63Tp+Cu1uh0A//Vui0aMaC/o1bVwVf\nBFQPdN7pdOsEpoEvDwhB76vQ/QT01UP6P7gX03ConAPb3rKC7zIiQvVFF7H50Uet4HuQuKjFj5Zx\nWY5zZlMzFBXqWWPVEfh7p9OiOTMFCvyQGi2E3bAX/ng4loJvyDUz/C50PQgZn4PAXKc3X6J/yJ7n\noPXL8Sf4FXOsVbIhqi+6iJduvZWzv/xl06FYRojnjyrq3rh9vcsp31yYCcVBR+zDCvaFHXO14pj2\n4heYOXylOhyztJRl4J/giH2kHXrXQ9dDEFrifkxDUTHbeuoYonLJEhpffJGeI0dMh2IZIZ7O8OFY\nHf+c0/Vc/8JM+G4z/NMepy2zT8HBPni1C8b54crs2K1VQAFvYsCDPniKU6tvWQnB2SBpoI44Pjqq\nC7K/635MQ1Fpxx2aIpSZyeQzz6R+9WqmrLBnIbzEcEYcfkREsqL3/01E7hWRU/WHNjxqyvRm+Gen\nwVcLIN8Ph/qgLQK5fvhaAfxuMhTE8C0z31RrpqRD1n9CyvngywEEfEWOx07uH+LPFx+goBh6e+Cg\nxik4lhPSX8e3eIvhyNXXlVJ/EZGFwFLg+8DPgLhwUKouhfs0nvn1iWOlsDDd8dXZ2uPYK/hw5t6e\nkea0b8aCLLLopZdOOkkjLTYXHS6BMgh81rnfux5Us1Pq6fgVBM90Mv94QiRa1nkbcl3c5LYAjuD/\nYflylFJIPLbuWgZlOILf74e6HPilUuoREfm2xphGhO4MH5y++/84AE+1O0LvAw5GnB/egnT4cr5z\nInesCHK0U6eEkrFfcKR0/gHafwL4op06yrnf8yykrIC0K9yP6WT01/FPXWo6kqSjcPp0VF8fzRs2\nUBht1bTEP8ORqUYR+QVOdv+9qK9O3Gz2utGaeU8bbO6Bf86D01OhKAD+6Frn74An2+GqcbFZq//E\nreuC3/smdP0NMr/q9N37BvyBOn4Hnb+OT8F/9znTUSQlIkJVdNatFXzvMBzh/iiwCrhQKXUIyAP+\nVWtUIyAnG9JSndZMXdzbCqemwqVZMDnoiH1XxOnRH+937scKY3X8vt1Ov33qimNir7qgbw/0bQFf\nHM4ytRu3Rqm56CK22Dq+pzhphi8iPuA1pdTRA1hKqT3AHt2BjYT+so6uXvwlGXBfm1Orn5vq1PB3\nh+HxI87IwwsyY7dWAQWs573YXXC4BE9zvrbe6Aw/IeAMMu99FyLNkHmD+zENRdkM57RtuBcCMeyP\ntQyLiiVLuP/aa+nt7CSY5vKek2VUnFTwlVIREdkoIqVKKUNTtodGd2vmFdmQIvBKF/ztiJPZ5/th\naYYz7DwUc9dMAxm+fzxk/xB6VkPn7yFy0OnJD86D1P8Dvqyhr+E2qRlOt07jZiizpz7dJnXcOIpO\nOYXta9ZQc/HFpsOxDIPh1PBzgXdF5CWgvf9BpVTcNODqPnyV64dP5sAncbL7oEBnBPaEYUeMxhz2\nkx+1SY4Qwef2VklonnPrp2+/MxSlZx2EFoIvhh9lYkX5LKesYwXfCDUXX8zmv/3NCr5HGFZbpvYo\nxkhNGdz/lP51/ngYbj8Eb3VBGCgPwqwUmJcGn8l1zNbGSgoppJJKG62Mw8CQiZ6X4Mi/Q+86wAf+\namfqVfeDkHolpJzrfkwno2J21Bv/StORJCU1y5fzxxUrULfeatszPcCQgh8ddRjX6M7wAf502GnL\n/MdcOC8DcqJj3Dd0w0d3OX36s2Jsoua64Pe+DR0/g9RLIPdukAF/oLabofO38Sn4q243HUXSMn7m\nTCK2PdMzDOekbZuItEZvXSLSJyKtbgQ3XAa2ZurizsOwOAMuzz4m9uBYJI8PODNvY4WxOn7fFojs\ngfRP4cywH4AvFyKGJnKdjAo7/cokInK0rGOJf4YUfKVUllIqO3pLBa4Afqo/tOGTOw5SQnpbM8uD\n8GwH7A8733dH4PVO+OZ+mJvitG3GCmPzbQPToG8ndD8F9EBfE3Q/CW23QHgDZN3kfkxDUVQJrQfg\nyCHTkSQtNRdfzOZHHjEdhmUYjHhXUCl1P87Iw7hC9zCUz+U5rpmf2g1lm2HiJvhYI+wNw//MhZKY\numbmmxF8/xTI+i50/Rn2nwLNc6D1BojshbSrIRiH3vM+n9Oeuf0d05EkLRWLF7P75Zfpbo2rD/6W\nQRiyhi8iKwd86wNOB7q0RTRK+scdLjxNz/WnpUBtCC7Lcko6E477ycXypG8BhWZ88UWc+n3obMhO\nc0zVvEBF9ADWzIWmI0lKQpmZFJ91FtuefJJpK1cO/QKLMYbTpXPJgPthYDtwqZZoxoAbnjp+gSkD\nWjCPDjUntrYOueTSRithwgTcdrBWCnz5A77vP0asYXJ7rKiYDdusN75Jai6+mM2PPmoFP84ZTpfO\n9aO9uIgZ7IzkAAAgAElEQVQUA3cBRTgmbP+tlPqv0V7vZNSU6XXNHAyfJv3z42ccObTQwnjG61nk\nRBwv6hI3tkknpnIOrPm96SiSmuqLLuL5H/zAumfGOcPp0ikWkftEZJ+INInIPVEhHw5h4EtKqenA\nWcA/icgH5uTGAjcyfDdx6vgGpl95kYrZTg2/r2/o51q0kF9biz8UYt87di8lnhlO+vYb4EFgEjAZ\neCj62JAopfYqpd6I3j8CrI9eI+b0b9rqbM10k4LoiVvLMMjMgewC2LPVdCRJy9Hh5rY9M64ZjuAX\nKqV+o5QKR293ACO2KRORcmAu8OJIXzsccrIhNUVvaybAzl447EIiWWBqvi0475o9L3jr3bNyjq3j\nG6Z2+XLbnhnnDEfwm0XkahHxR29Xw8hSTxHJBP4KfCGa6WvBjbLOl5scAzXd5JvO8FuWQ2SfufVH\nihV845Sfdx5733iDzpYW06FYTsBwWkA+BfwE+BHOCKTno48NCxEJ4Ij9b5VSD5zoeTfffPPR+3V1\nddTV1Q13iaP0C74u10xwWjM39ei7fj/GfPHB2bgNTIPwescx0wtUzoEn7jAdRVITTEujfNEitjz2\nGLOuusp0OAnH2rVrWbt27ZiuMZwunQZgLM6YvwbeU0r9+GRPGij4o8WNDL8mBE+0D/28sWJ0vi1A\nYLoj+Cl17q89Girn2GEocUDNhz7E5kcesYKvgeMT4VtuuWXE1zih4IvIrTgZ/aAopT4/1MVFZAHw\nCeBtEXk9er0blVKPjTjSYVBdCvc8ruPKx6gNwW0ufGI1Pt+2P8P3CkWV0NYCbQchK9d0NElLzcUX\ns/rGG4mEw/gCLp8hsQzJyf5GXhlw/xZgxEYqSql1gH/IJ8YItzL8TT16Z+j2U0ghzew3J/jdHtqA\n8/mgbKaT5c9eZDqapGVcSQnZxcXsevFFShcsMB2O5ThOKPhKqTv774vIFwd+H6/02yvoFON8v7PT\n3dwHhZoTGGMmauC9DB+OlXWs4BulJtqtYwU//hjuMUpP9OdlZ0JGOuzR2M0ocizL140xm2QAfwmo\nVogcNrP+aLAWC3FBjW3PjFs8cG5+ZLhR1qlNgc0uCb6xXnzxQWCKY4vsFezGbVxQPH8+rY2NHN65\n03QoluM4oeAPHHwCzB4wBKUt3gagDKSmFDZt17tGbQg2detdA5ySTgstRIgM/WQd+KdC+D0za4+G\n8lmw411rsWAYn99P9bJl9tRtHHJCwT9u8ElgwP0spVS2m0GOBN2++OCUdNzI8EOEyCCDQxga7hGc\n7q06fkY25E6A3VtMR5L01CxfzpZHHzUdhuU4bElnFLh1+Ar6N24NlXW8uHFbMRvqbR3fNFXLlrF9\nzRrCXXE3OiOpsYI/mjVCsKXnmB++ToyaqHlW8O2MW9Ok5+czYfZs6tesMR2KZQAJJ/jVZbB1J0Q0\nlr2z/JDlg91hfWv0Y3Tj1l8FfbtAeShLsxu3cUPtJZew6aGHTIdhGUDCCX5WBuRkQWOT3nVqU9zy\n1Ck014svQQhUQniTmfVHg23NjBtqP/QhNj38MMpLrqsJTsIJPrizcVvr0sat0V58iJZ1PNSaWVQJ\nrc1wxNBGt+UoBdOm4Q8GaXrLfuKKFxJS8GvK9Ldm1oRgowutmeMYRyeddOPCYoPhtTq+32+z/DhB\nRKhdsYJdL7xgOhRLlIR0N3KrU+e5Dr1rAPjwkUceB2hmkp5hYScnMA26PFaHrZrrCL61WDDOBT/4\nAT6/a3ZaliFI2AzfDcF3o6QDUGCyju+1DB+gci5se8N0FBawYh9nWMEfJVUh2N4LYRf2o4wOQ/FP\ncTZtlYdOr9rpVxbLoCSk4FeXQn2j3hP2qT4oCjiir5sCk66ZvgzwF0HfNjPrj4byWbBzPYRd+Mux\nWDxEQgp+ehrk58DOvXrXqQ3BZpc8dYwJPhybfuUVUtOhsBR2bTQdicUSVySk4ENiWSz0t2YqUy7V\ngWneMlEDp6yz1dbxLZaBWMEfyxouCX4aaQQJ0oYhk9LAdAi/a2bt0WLr+HFLb0cHmx5+mLfvvpt9\n77xjOpykIiHbMsGxSdYt+FNS4JEjetfox+nUOUA249xZcCCB6dDxU/fXHQsVc+CBH5uOwnIcnS0t\nPPSZzxBMT6fz4EHeBhbddBOTzzjDdGhJQcJm+LXl7pR0NrrWmlnAAaOumRtAGfLlHw39vfgW42x7\n8kl2/v3vAGxZtYrU3FyW//znXPXww1QtW8Zrv/qV4QiTh4QVfDdKOqVBaApDpws6aHTj1jcOJAf6\nPDTBKH8S9IWhRfPOvWVIdj7/PG/eeWwkdriri1BGBuHubnrb28maONFgdMlFwgp+ZQns2A1hjY6W\nAYGKoGOVrJsCCthvKsOHaB3fQxu3InDZF6Gn03QkSc+0lSvZuW4d4Iw/DHd18ecrruC5f/93juzd\ny5xrrjEcYfKQsDX81BQoKnBEv6pU3zr9J25npepbA5wavnkTtfXAReZiGCmf+LrpCCzA+JkzOeXT\nn+a/zziDolNPZdIZZ3Bo+3Z6OzuZeNpppObmmg4xaUhYwQenjr9pu2bBT3Gnjp9LLnnkEyGCz8QH\ns8B06H3J/XUtCcH8L36ROddey7533mHtTTfR9OabiN9P6YIF7HjmGdLy8jj/e98zHWbCk7AlHUgs\nTx0/fq7lejNiD9701LHEFeLz8erPf052cTErbr+df9m5k4/ddx/n3HgjG++/n/Z9+0yHmPAkvODv\n0j0IJQSbDDkXu0owWsO3wywso+TpW24hr6aGy++6i6mXXUYgJQWA7OJiMsaPZ++btqtKNwld0vn8\n1eDT/Jbm5kBzo/gKnAlYkT3gn2Q6GosHaWtsZPzMme97rHnjRl75+c+pWb6cqvPPRymFiBiKMPFJ\naMHXLfbgGKh1KjjUBzmJ7gTb76ljBd8yCmZceSV//8//pG33blobG9n62GP09fQw7cMfZuaVVwJY\nsdeMxMO8SRFR8RDHaDltG/y0COalm45EM4f/FwRmQsY/m45k+PT1wYFGGB/duX/1cdj8Ksw8B2Ys\ncNo3La6gIhEONzSw4YEH6D58mOqLLrInbMeAiKCUGtE/YCv4MeDju2B5JlydYzoSzbT/F4Q3wrjb\nTEcyPNoOwh+/A4f3w+F9MHcprLvXOZTV2QbX/7tzItfiCicq1/T19iI+nx2WMkJGI/gJvWnrFm6Z\nqA2kl17aaKWbbjpwYdYieM8186m7oKkeLvw0fOG/4aWH4dyPwNf+DKXT4Y3VpiNMKkSEzoMH2fuG\n42Ia7na6HfzBoBV7l0joGr5b1IbcM1HbzCbW8x711JNNNoc5RB75zOcsKqgkSFDf4l5rzTywG6bO\nd8o3AHMWw7vrYP6l0H4Y0jLNxpeERMJhXrv9di6+9dajXTodzc20799Pfm0tkd5eAqmaTzEmMbak\nEwNe7oT/uQdeq9S7zm4aeZEXyCOfAAH2sJtzWEQ6adzLPZzF2dQyRV8ASkHTOBi/A3weOB358qNw\nzw/g8i9Bbxes+jWUToPWA05Z54JPwaQq01EmLa27drH50Uc5vGMH4a4udv3976QXFnL6Zz9L9bJl\npsOLe0ZT0kmKDH9HI4RCMLFQz/VrooevlNK7B/gKLzOJycxjPgBrWM0WNrOAhRQxke1s1yv4Isey\n/NDZ+taJFTPPcaZevfMMhFLhguuh+lR47h4I91ixN0B/Hb/p7bdZ993vkjlxIrlVVeSUl3PKpz5F\nR3Mz9193HV/Y5qGRmh4ioQW/X4D/8VuOr45S8NXPQMVkCMaw8pHjhzSBPWGYpLGiMoEiGtnFEY7Q\nQTv72Uc55QCUUOKOm2a/iZoXBD8tEy7/Itz1DXjgVsjMgdQMyMiB8pnww0/BJ2+BwhLTkSYNIkJv\nZycv33YbebW1nPYP//A+t0ylFONKSuhsaSEtL89gpIlJQgt+P23tcPlSWHgq3HIbpKXC1ZfAKdNg\nXFZs1ug/gKVT8Kuo5gDNrONZBKGKak7jdDroYDwTmMJUfYv347XpV2+sdko4/3qXk/FnDBggc/Ol\n8PpTcMF1xsJLRoJpaWz+29/4/JYt+EMh+np6aNm6ld0vv8wLP/oRtStWkJKdbTrMhESr4IvI7cCH\ngCal1Gydaw1Gb69TyikpckS/pAg+dzWsew1Wfh6++Tn450/EZq3aFKesU5cRm+sNRgEFXMRyHuIB\n3sMR3Sd5nAwyqaCSKUyhmhp9AYAj+O1P6l0jljx3D2TmwrwPHXusrw86WiElHToOm4stiZl2xRXc\nf911+Px+xpWX09fdDSKc+/WvM23lStPhJSy6M/zfALcCd2leZ1A2bodVz8GOPfD8G/DCm3CkAzLT\n4cPLoDqGLpq1IdjogqfOduoJEuJ8LqCIiUc3cOvZxiM8zKf5DJlo7D4JzPBWhj91Pqz+rZPpl81w\n+u93vAtvPwPFtbD0WtMRJiWLv/Uttj7+OG27dxNITSWUmUlORQUTZs+mbfduUnNyCKYn+klG99Eq\n+Eqp50SkTOcaJyMchj374aKFsH4b/Oir4PdBXo6T5Pn9EInExoKhNgR/d6Edfg2rOY3TmcOxA0MK\nRS1TeJq1tHBAr+D7S0Edgkgr+DzwsXvehxyRf/Yv8Mf/C4ebIXcCzF0C533CqetbXCeUmXk0k+85\ncoT61avZ9NBDPHHDDRzZu5ea5cuZ/pGPULpggeFIE4uEruGfMt253f0I/OoeuP5rUL8Lmg/Bkvnw\niQ/B8kWxWWuKS/Nt00ijnfb3PdZNN2/yOhOYQBYx2pQ4EeKDwNToxu18vWvFgqxcuOQfobnR2cHP\ntz5A8cQbd9zBs9/5DnnV1VRdcAEX33YbE2bP5qXbbuOZb36Tq1etMh1iQhE3gn/zzTcfvV9XV0dd\nXV1MrvvnR+GhtfCVT8PUSmcoyqTx8OKbcOUNsHies4k7VqpCsL0XwsoZfaiL0zidTWzkr/yZFlo4\nxEEAyqmgjsXk4kJnQ7+JmhcEHxyxz5t47KNcX58zkN3nd8dhzzIoR/buZfuaNVx2552UnHXW+36v\nYOpUNj/yiKHI4pO1a9eydu3aMV1D+8GraEnnoZNt2uo8eDVzBdzxHTh9gCtrfxnnjI/C774HUypi\ns1bFZni8FGpSYnO9E9FFF/VsI0iIQgoZx7HOE4VC0GwIduR7ENkP2T/Qu06sWP17KKqE6Wd98LCE\n7sMTlhPSceAAP50xgxv27iUSDtPb2Ul3aytNb73Fq7/4BXOuucZu4J6EeD14JdGbEUqK4O9vOFn9\n+DwI9zlzbn9zH5wxE8bFsNzdP/1Kt+AHCDCN6e97LEIEif7STmAadDytf51YUTnHOXgFjri3tThZ\nfzDF2bi1GCE9P5+KxYv50+WXM/G00/AFgzS9+Sbhri7mXncdUy+7zHSICYfWDF9E/gDUAflAE3CT\nUuo3gzxPW4b/2LPw8NPQ2QX7WqCxCQIBWDofvnw95Mdwz+5ze5zSzhfzY3fN4+mjj1d4mXnMp48+\nBHF/7GF4C7QshfHb3V13rOzd7hiq7d8JgaBT0tm/E5ZcAwttJmmC1sZG9r3zDge3biXc3U35okVM\nPPVU2vfvp72p6QMDUyzHsPbIJ6C3F5591blfUQxlk46VbmP5if7WFnivG342cejnjoUX+TuncBoh\nQkcfCxMm4NaWjOqDvVkwYR/4PGJA9urjcPe34fQLoWwmZOdDJNqPf8fX4EfPO6dwLcbYv349b955\nJ+vvuQfx+5l85pkE0tJY+JWvkFup2ajKg1jBH4RwGFqPOK2YA9FRul11BL5/AJ50qRH1IC3sYMfR\nrp1UUimgkGKK8aPZbnb/XBj3KwidrnedWPGtK+C8q+Dsyz+4UftPp8LnfwFT7DAOU7z6y1/y5p13\nMunMM5l99dVMOu00ultbefyGG8iYMIHF3/qW6RDjjnit4RulvRN+/7Bzwranx+m99/v17NNNcenw\nVZgwr/EqW9hMPvmkkooPP9vYygbWM5s5zELzweajnjoeEfzmXc7JWp8Pujpg3w5oeA+e/pNjqFas\n0XTOclIannuOLY8+yorbb6dgqmMPopQiJTubwunT2fXCC4YjTBwSXvDHZUFB1Mk3dKwCQk/P+7+P\nBSVBaO6D9ghkaCyr72Y3m9nEmcyjiCLSyTia0a/nPV7kBZcE30Pe+J+4CR77Fdz+v6F8FqRnOxu5\ntadD3cchwwOHyBIU8ftp27PnqNiDk72+fffdNL39NnUDWrYtYyPhBR/g48udr0/93bFXaD4EqSGo\nLoO5U2HOFGcjd6z4xdm03dwDczXOcOignU46qeFYh0kvvbRymJ00UEoMPSNORGAadN6pf51YcebF\nzpSrzjY4chC6OyE9C2rPcDZwLcbo78Ff9eUvk1lUxLbHH2fPa6+RX1vLWTfcQP4U++krViR8DR/g\n4GG49fewvRGmVToZfyAAT70Ae5vhq/8D6s6MzVord8KV2fDRcUM/d7S0084D3EuAIBMowo+fbro5\nxEH66ONCLiIbjQEA9K6Hgytg/Ga96+jgzbWwc73jorlvBzRuhmu/dWwylsV12vbsYffLL1O/Zg2F\n06cz/YorrD3yENhN2xPw2wfhgdXOaduKYsjJOpbRf/OnTrvmT/4tNmt9tckp53xd07CVfpppZhMb\niRBBoQhFxb+cGJ0iGwrV63TqFB0ESXNnzbGyfyd8+8OQlgU1p8H4MhhXAKE0+N3N8JU/QInNJuMF\nFYkAIPY09KDYTdsTsHufc8DqjFnHHuvtdTL+vc2OL36sqA3BWhdM1AqivwD2sY9OOukjzJu8QQkl\n5KHxMACABCFQBeFNEJyjd61Ycf+PYdGVsPJfPvh7T9wB9W9ZwTdMf+InIlboNZAUgn/ROfDqu/DR\nfzk25vDwEcdIbUqFMwwlVkxJgV8eit31TsZrvMJrvIYgpJBCBCcj2sZWZjOXKjSP8Osfd+gVwd/X\nABOjP5P2Vuhqh4N7Yd29Tl3fnro1jlibC63ETUknEolo/cveWA+PPO107YSCkJ0Bc6ZC+eTYrrM/\nDLVboGWKXouWnTTwEi8yg5mUUkY6x7zDn+NZ9tHESj6sLwCAtm84X7O+qXedWPHmGvjTd53WzLlL\noPOII/TBFFh6jTP20GLxCJ4u6TQ3d1BYqO+k45QK59bYBG9tdAahvPaec5s/x/HaiQUF0fNOzX1Q\nqPGnu599dNPFVI7Vo3rppZtuOuggDRfq6oFp0HWv/nVixaxFjonanq1OD35mLpyyFGYuNB2Z5Tja\n9+0jLT8fn1/zAcIkI24Ef/PmFq2C39kFN98G72yGwjxIS4HObti2EypL4Bufdb6OFRGnrLOxR6/g\nl1DGq7zKWtZQTgUCtNLKHvbQRScLcKHjJDAdwh46AenzwYQy51Y1Fza+BO88C3/5D2cQyhU3OFm+\nrR0b53fLlnHRrbdSutC+GceSOBL8A5x9dgwU9wT8ZRU0HYB//RTMqHZEv5+Pfckp93zu6tis1e+a\nuVDjhLYCCljK+dRTz6u8TBddZJBBKaXMZPb7fHa0EZgC4W2gekBcWC9W3P4VWP0753Rt9Wlwzkcg\nGIKffBau+RbMXWw6wqSn9pJL2PDAA1bwY0zcCP6mTQe0Xn/NizCr5v399kfaoeUwBANObT9WuDHf\nVhAqqKSCY6ZSHXRwkBYaaaSUUv1+OpLqjDwMb4bgDL1rxYoHf+LU7e+od0R+IJtegQ0vWsGPA6Ze\ndhl//djHOP8//sNu5MaQuBH8zZtbtF7/siXwsz85PfdnznJ88ZsPwrrXHZ/8WI06BEfw/9Qau+ud\njHrqWcezNLILgFzyyCOPDaznVE5lAkV6AwjMcDx1vCL4bQcdT51gCMK9zinbSATeex52bYRzNG90\nW4ZF0SmnEO7upnn9egqnTx/6BZZhkTSCv2whpITgxbccq+SeXufE7f/6GJwbY/+v/hq+bnbSwBu8\nxkxm8VGuPFrG6aOPh3mQd3nXBcGfDuF3gY/oXSdWnL4MfvEluOPfnDp+42bY8S50dzieOud93HSE\nFpwOlCmXXsqG+++3gh9D4kbwt2xpQSml7eNbagpceI5zA2g55DhpHjgET78Mi2LojFsdgq090Kcc\nfx1d7KaRCBHmcsrRHnwAP34yyeIIbfoW7yc4A7ru179OrKg9A77ye3jhQXj+fsgYB7PrYPrZUDJ1\nyJdb3GPqpZey+mtf45wbbzQdSsIQN4Kfmhpg794jTJwYw2L6IPzg13DXg9DV7dTuAaZVwZN/h//9\naciKQaNQhs9pz2zohQqNe5mFjOd1XqORXUxkEh10sI8mtlNPJx3M56yhLzJWAjMg/B3968QKEadL\n59LPOTdL3FK2aBEtW7bQumsX2cXFpsNJCOJG8Gtr89m06YBWwb/jPti2C/7rRqeOnz6gVf3Mj8Hi\neXDevNisVRuCTT16Bb+EUhZwDi/yAg000E0XGWRSSimncCrjmaBv8X4CUyC81fHWEY+5Tip1bBKO\n3RiMO/zBILUf+hDr77uPeZ+zb86xIK4Ef/PmFhYtKte2xj1PwMcufH+nTm+vs4E7Pg/2x3AboTbF\nEfxlsbvkBwgSZBazKaeCIEFS0ejJfCIkFfzFzpzbYAxNidzACn3cM3XlSl788Y+t4MeIuBH8mpo8\nNm/W25o5bzbc95TjpzOzxhl9uHUnPPG844sfyzq+G62ZAApFFsc+FfXX8iX6yxUCM5yNW68JviXu\nqbrgAu6/5ho6DhwgPV+zIWASEFeCf/fd72hd41MrYdVzcPffYP02aGt3Mvvzz4brLoMJBbFba0oI\nHjsSu+udiONF3YeBU6L94w4tlhgTTEujculSNj30EHOvu850OJ4nbgS/v4avk0nj4fqVTqdOaghy\nj5sREsvB5rUutWbGBYHp0P2Q6ShGTk8XNG233TlxztTLL+e9v/zFCn4MiBvTkOrqPLZuPUgkot+9\nc2LhMbGPRN6/dxcryoOwNwydkaGf63mCHptv20/TDvj6xaajsAxBzfLl1K9ZQ097u+lQPE/cCH5G\nRoj8/DQaGg67uq7Pp2fvLiBQEYQtyZDlB6Y69goqbDqSkTGpGg7tgyMuDTCwjIq03FyK581j66pV\npkPxPHEj+AA1NfnaN27dpL810w1+x10cxt03y6NIOvgnQl+9mfVHi98PFbNh25umI7EMwdTLL2fD\n/R464BenxJXg19bmaa/ju0l/a6Yb9NLLfva7s9hgeHXjtmoubH3ddBSWIZiyYgWbH3mEvt5e06F4\nmrgSfCfD1+up089fHoPXNevTFJdaM8GxS25mnzuLDUZgWtRTx2NUnWIF3wNkFxeTW1XFjmeeMR2K\np4kzwc9zTfCfew2eekHvGm6WdAoopJlmdxYbjIBHN24r59qSjkeYtnIl6+/10IS1OCSuBL+6Wv/h\nq35qy2HzDr1ruOWaCY7gGy/p9Howwy+fCY2boMelj2KWUTNt5Uo23HsvKpIMrW96iCvBr6rKo6Hh\nMOGw/r/Q2nLY3KB3jfF+CCs44ELzSiEFHDCa4U+Dvo2gPPafMSXNmXO704OfTpKM/Npa0gsK2PWC\n5o/mCUxcCX5qaoCiokx27NDfJldTBpu2611DJGqx4EKWn804Oumkiy79iw2GLxskD/q2m1l/LFTO\nsXV8jzB15Ureu+ce02F4lrgSfHBv47akyPHCb+/Qu86UFGe+rW58+Mg3nuV7tFPH1vE9w7SVK9lw\n330opf+AZiISh4LvTmum3w+VxbBFc1nHLRM16O/UMSj4wRke7dSxrZleYcLs2QA0vWnfoEdDXAp+\nIm3cut+pY3Ljdoa3M3ybNcY9IsLUyy9n/X33mQ7Fk8Sh4LvXi+9GHX+Kq4JvOMMPeDTDzyl0Bps3\naX73t8SE/rKOZeRoF3wRuVBENojIJhH5ylDPd6ukA06Gv0nz//GaFMdPxwVPOAooMN+aGd7gvU4d\ncMo6294wHYVlGJScdRbt+/bRsmWL6VA8h1bBFxEf8BOcwU8zgI+LyEm9aCsqcmlsbKOnp09naICT\n4esu6WT6IM8Pu4bZmrl27dpRr5VPAS0ceN9Ac1fxZYPkQt/Yf6hj+TmMijjeuHX9ZxHHrF27FvH5\nmLJihfXWGQW6M/wzgc1KqR1KqV7gj8ClJ3tBKOSnpCSbbdsOag7NnRo+jGzjdiz/uVNIIZ0MDmPQ\n/TFGG7eui1wcWyxYwT9G/8/ClnVGh27BnwzsHPD9ruhjJ8Ut18yiAujsgkOtetdx00TNfB3fo62Z\n1fEr+JYPUrF4MfvXr6e1sdF0KJ5Ct+AP5jI/ZDV7wYISurv1l3RE4MKF0Kz5w8QZqbhWZCmjnF4M\nOgoG5wH6/+5iTlEl5E2C7k7TkViGgT8UYvbVV7P/XQ82CRhEdB5gEJH5wM1KqQuj338VUEqp7x33\nPNsPZ7FYLCNEKTWi0U26Bd8PbASWAHuAl4CPK6WscYnFYrG4jNYh5kqpPhH5Z+BxnPLR7VbsLRaL\nxQxaM3yLxWKxxA9GT9qO9FBWoiIixSKyWkTeE5G3ReTzpmMyjYj4ROQ1EXnQdCwmEZFxIvIXEVkv\nIu+KyDzTMZlCRP5FRN4RkbdE5PciEjIdk1uIyO0i0iQibw14LFdEHheRjSKySkTGDXUdY4I/mkNZ\nCUwY+JJSajpwFvBPSfyz6OcLgAf7O2POj4G/KaWmAXOApCyJisgk4HPAqUqp2Tjl6CvNRuUqv8HR\nyoF8FXhSKTUFWA38n6EuYjLDH/GhrERFKbVXKfVG9P4RnP/UQ55XSFREpBi4GPiV6VhMIiJZwDlK\nqd8AKKXCSinNp0biGj+QISIBIB3YbTge11BKPQcc30B+KXBn9P6dwGVDXcek4I/qUFaiIyLlwFzg\nRbORGOVHwL8yjDMbCU4l0Cwiv4mWt34pImmmgzKBUmo38J9AA9AIHFJKPWk2KuOMV0o1gZM0AoVD\nvcCk4I/qUFYiIyKZwF+BL0Qz/aRDRJYDTdFPPMLg/06ShQBwKnCbUupUoAPnY3zSISI5OBltGTAJ\nyMug35cAAAQqSURBVBSRq8xG5T1MCv4uoHTA98Uk0Ue044l+TP0r8Ful1AOm4zHIAmCFiGwD7gbO\nE5G7DMdkil3ATqXUK9Hv/4rzBpCMLAW2KaValFJ9wL3A2YZjMk2TiEwAEJEiYN9QLzAp+C8D1SJS\nFt1tvxJI5o6MXwPvKaV+bDoQkyilblRKlSqlKnH+TaxWSl1jOi4TRD+u7xSR2uhDS0jejewGYL6I\npIqI4Pwskm0D+/hPvA8C10XvXwsMmShqPXh1MuyhrGOIyALgE8DbIvI6TmnrRqXUY2Yjs8QBnwd+\nLyJBYBtwveF4jKCUeklE/gq8DvRGv/7SbFTuISJ/AOqAfBFpAG4Cvgv8RUQ+hfOG+JEhr2MPXlks\nFktyEHcjDi0Wi8WiByv4FovFkiRYwbdYLJYkwQq+xWKxJAlW8C0WiyVJsIJvsVgsSYIVfItnEJG2\n476/VkRudXH9eSLygoi8HrUq/oaJOCyW0WLs4JXFMgoGOzQy5oMkIuJTSg1nzvydwIeVUu9ET3tO\niWUcFotubIZvSQhEpFREnhSRN0TkiajFMlGnyZUDntcW/bpIRJ4RkQeA90QkXUQejmbvb4nIYKcW\nC4F+d0KllNowSBwFIvJXEXkxejs7+nh6dIjFiyLyqohcEn38WhG5X0TWRIcBfSPmPxyLJYrN8C1e\nIl1EXoveFyCXY/5LPwHuUEr9TkSuB24FLh/kGgMz8VOAGUqphuibQqNS6kNw1Iv+eP4fsFFE1gCr\ngDuVUt3HPefHwA+VUs+LSEn0edOBrwFPKaU+HZ1M9JKI9Nv7noEzBKgLeFlEHlZKvYbFEmNshm/x\nEh1KqVOjt1Nw/ET6OQvHXRPgtzium0PxklKqIXr/bWCpiPy7iCxUSrUd/2Sl1LeA03D8nz4OPDrI\nNZcCP4l6Ij2IY+ObAVwAfDX6+FogxDG32CeUUoeUUl04LpALhxG7xTJibIZvSRSOr6H3fx/m/YnN\nwDmo7UefrNRmETkNZ9LWt0XkSaXUtz+wiFL1wC9E5FfAfhHJPe4pAsxXSvW870ERgCuUUpuPe3z+\nSWK3WGKKzfAtXuJkw1Cex8m6Aa4Gnove3w6cDiAilwHBQS8sMhHoVEr9Afg+g/jOi8jFA76txXkz\nOXTc0x7Hcbjsf82c6N1Vxz0+d8BrzheRnOg0q8uAdSf6Q1osY8Fm+BYvcbLM9wvAr0XkBmA/x2yE\n/xt4IFpKWcWArP44ZgHfF5EI0AN8dpDnfFJEfogzeSoMXKWUUtHsfWAct4nImzgzWJ8B/hH4NvD/\nROQtnDeuemBF9DUv4ZRyJuMMwLH1e4sWrD2yxWIQEbkWOE0p9fkhn2yxjBFb0rFYLJYkwWb4FovF\nkiTYDN9isViSBCv4FovFkiRYwbdYLJYkwQq+xWKxJAlW8C0WiyVJsIJvsVgsScL/B00TjfT5Rla6\nAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fea577cb610>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#Contour Plot:\n",
    "yy = np.dot(hoursStudy.reshape(100,1), np.ones((1,100)))\n",
    "xx = np.dot(hoursSleep.reshape(100,1), np.ones((1,100))).T\n",
    "\n",
    "CS = plt.contour(xx,yy,100*allOutputs.reshape(100, 100))\n",
    "plt.clabel(CS, inline=1, fontsize=10)\n",
    "plt.xlabel('Hours Sleep')\n",
    "plt.ylabel('Hours Study')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAADtCAYAAAAcNaZ2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsnXmYFNXVxn/39t6zMiwzwLAIiCgugLgbcN9FjQKiMYoa\nNYkalSiaaBI1Jpjoh0TBGBVxiSYaoxgDGnGPu3FXXFAEAdln7Z7e6t7vj+rqqWl6ZnqbgZF6n6ce\nYOiubareOnXOe94jtNY4cODAgYPugdzaO+DAgQMH2xMc0nXgwIGDboRDug4cOHDQjXBI14EDBw66\nEQ7pOnDgwEE3wiFdBw4cOOhGuDv5f0dP5sCBAwe5Q7T3H06k68CBAwfdCId0HThw4KAb4ZCuAwcO\nHHQjHNJ14MCBg26EQ7oOHDhw0I1wSNeBAwcOuhEO6Tpw4MBBN8IhXQcOHDjoRjik68CBAwfdCId0\nHThw4KAb4ZCuAwcOHHQjHNJ14MCBg26EQ7oOCoZSikQigTNvz4GDztGZy5gDBxmhtUZrTTweJxaL\nkUgkEMI0VnK5XHg8HlwuF1JKpJSp/3PgYHuHQ7oOcoKdbEOhEFJK3G43QgiklESjURKJBIZhtPme\nlBKXy5VaHDJ2sL1CdPJK6LwvOgDakq1SCoBwOIxSCsMw0FqnCFQIgcfjSRFr+jrscMjYwXcU7V7A\nDuk66BBa61TOVimFEAKlFNFolEgkgsvlIhAIpCLbWCyWImClVOrvFplaxGonVfvnLDhk7KCHwyFd\nB7mhPbKNRCLEYjG8Xi9gkqPH4yGRSKTSC0KI1P9b60lftNYpIrUvFqlaUbFDxg56KNq9IJ2croM2\n0FpjGEZKjWCPbGOxGD6fj4qKCqSUtLS0bEGK1josCCFSBJn+GTsJW2mLTGQshGhDxlbxzr5dK5r2\ner0pYrZ/z4GDbQUO6ToAMpOt1ppwOEw8Hm9Dth3B+l5nKDYZJxIJIpFIGxUFsEVU7JCxg60Nh3S3\nc3RGtn6/n2Aw2CnZFgv5krGd7O2kml7os/60k7E9z+yQsYOuhkO62ynaSyO0tLSQSCTw+/2UlJR0\nSkLd1RDRGRnH43GATiNj6+GRSXUBOGTsoMvhkO52BisnaqkNMpFtaWlpViSzLRCRRcZW0S0QCAD5\npyksMk6HRcJutzvj9xw4yBYO6W4n0FqTSCRIJBI0NjZSWlqK1ppIJIJhGDmRbU9AR5GxYRgpMrbU\nGdCqjEgnZIvADcMgFou1WZ/1ebfb7UTGDrKCQ7rfcdjJ1o5QKITWukvIdlv2YBBC4Ha3veytKNki\nYsMwUpGxXWPcHhlHIhEAPB5PahsWgTuRsYN0OKT7HUUmsrUq/Eop/H4/gUCgIBLIpFToiaRikWF6\nsTBbMrbOQWeRsUPGDsAh3e8c2iPblpYWAPx+P1prPB5P0W707yphZEvG1r9DoVBWkXF7ZGxPUThk\n/N2FQ7rfEVg3c3tkGwgEUkQbjUaLuu1MngrfZaSTsaWA8Hg8BaUp0n8v6dI2+/cc9Fw4pNvDYRWD\n7BX3eDyeyjPaydZCtg0M2cAwDJqamlBKtSEDqy14e4rWCk1TZCLV9AeptZ30FIXV9OFg24dDuj0U\n7ZFtS0sLUsqMZFssaK2JxWKpNuBgMJhSAFhytI4kWtsbQXQlGVsSQMvVzU7GmcyFHGx9OKTbw2DX\nnFqIxWJEIhGklJSUlKT8bdtDvpGuRbZWFO31ekkkEng8HmKxWKqolEkva8m07AY6HRnebA/ojIzT\nz1kmxzYgJfmz/p7eXWgnY8ckaOvDId0egExettBKti6Xi5KSkpRkqSu2byfbYDCI2+1u02TRHjLp\nZfOVaG0v6IiMMzV8AKk3nPbOWToZA6nPOWTcvXBIdxuGnWybm5vxeDx4PJ6Ul63b7aa0tHQL3Wm2\n687mM1YaQUqZIlu7WXk+EXO+r9uZGhe2J4LI9AAzDINIJILP58uq+649MrbDiowd+8yugUO62yDS\nvWytn8ViMcLhMG63m7KysrzIFjqXeKWTbbYpi0KRa4RnJxX757qbjK392FoohmNbOqkmEomUn4UF\nh4yLA4d0tyFkMg7XWhONRonH47hcLsrLy7e4uYq5/VzJtjuQDalY58wq7m0vxbtMfsYWuoOMrbcT\nn8/nkHGWcEh3G0B79oqRSIRoNJpKK1i5t0KRnhbIl2yLKT3LB3ZSsfY1EAhskaJwindtUUwytkjY\n5XI5kXGWcEh3K6IzsvV6vanINhwOd8n2uyKy3dqNEnZSsYqLTvGuc+RDxtY5snTZmSLj9Ckf2zsZ\nO6S7FZCJbMGcrmvNH0uf0lDsqDKRSNDQ0FAUsk1vvNgWUYhWdltuze0ovVAsdETGsVhsC2kb0O5D\nzF4c3l7J2CHdbkR7kW1LS8sW88e6avuxWCzVbmopH75rF3UuKKR4Zy3p3XjbC6zzZrnVwZYPsWy1\n2Z2RcXrDR08mY4d0uwHtkW0u88eEEG00urlu355GsCb1Fqrr3do53a5Erq/asViszSu2ffkuIz3S\nLmb3nZ2MY7HYFttK1xj3lPl3Dul2ISzHr/QZXfnMH8vnQsqUs/V4PCnj8mLBupm6SlWxLSETGbe0\ntKQiMYtUYrFYt+SLuyO9UAx0JRlHIpHUFOj333+fZcuWcc4552ylI+0cDul2AazXpHA4jM/nS0Wp\nuc4fy7TebD+XiWyLDevCb2pqwjCM1I1lRYA9/TUwF2yvnXeFkn4xyNje6rx69Wo2bdpU6GF1KRzS\nLSKsyNZKI7S0tODxePKaP5aObF7lsyXbQtMC1kPF0sRaHr3QWq1u7ybZnkbaFEoo2Zyr73J6J9tz\nB+bbxpQpU5DS7JwcMGAAo0ePZvTo0al0Wkc455xzePLJJ6muruaDDz4AoK6ujqlTp7JixQqGDh3K\nww8/TEVFBQAXX3wxixcvpqSkhAULFjBmzJisj+27nXDqJlgkFIlESCQSbfKvTU1NeDweKisrC57U\n0NH2o9EoDQ0NRKNRSkpKKC8vL3p0ax1nY2MjLS0t+Hw+wDS+sW4QqzBnpU5KSkoIBAKp7jmLrEOh\nEOFwmEgkksqHWoWq7zrs58rr9bY5V36/P+dztTUfXt2d3kg/d9Y1GAwGueWWWxgzZgzl5eX861//\n4owzzuDjjz/Oar3Tp0/n6aefbvOzWbNmcdhhh/HZZ59xyCGH8Pvf/x6AxYsX8+WXX/LFF19wxx13\ncMEFF+R0DE6kWwAyGYcbhkFLS0uKdIvVQZYpOs03jZBPpGsRgFKKQCCA1+tNRfOdbStbdYAVtaRX\nqbf11+5iPSjy1cla52ZblbV1B6SUjBw5kmAwyNlnn82RRx6Z0/cPPPBAVqxY0eZnCxcu5MUXXwTg\nzDPP5OCDD2bWrFksXLiQH/7whwDss88+NDQ0sG7dOqqrq7PalkO6eaC9KQ3W/DGLlOrr64t68Vs3\nd3flbCEz2VrHVAjZFJoD3dYq1V25H+2RsVKKpqam1DmKRqOpoq11njweD16vN/Wn/fdXDGztQl76\n9puamqisrCzKutevX58i0pqaGtavXw/A6tWrGTRoUOpzAwcOZPXq1Q7pdgUyGYenzx+zX9TFlFTZ\nu9Xs3rn5kG02+5VIJAiHwxnJNpf15LpfneXxrIedJSGyy7K2luFNodBas3nzZlavXs23337LunXr\nWL9+PRvXrGHzurXUbdpEQ309jU1NNIfDhCJRNoVC+IEY4HW78UiJFMJcrPUCSmsMrUkoRVwpEkrh\nkZK4UriBipISelVWUhIMUlJaSllZGeWVlVRWVdGrb18qe/WioqKCXr16pZbevXtTVVWVGk+0NZFO\nuo2Njam8a1duMx25XG8O6WaBzsi2vSkNxSIlK5dqSZG6MrK1jssyxrbUF1sTdjK2O6t1ZnizraUo\nNm3axNKlS/nss8/45IP3ueueBdSUldIQieKVkoE+L/2loNpI0DcWZYBhsCtQJaBCQDmQ0HCJIQgB\n04Vgqta406wZO4IB/EEpngH2kZKjQiGMUIgoELEtzcAGoMXrJeLxEHa5CAHNWtMYj1MfiSCAcr+f\nHYYOpV+/ftQMHEj/QYOorqmhurqa6upqapJ/t3KvXY2GhgZ69epVlHVVV1en0gZr166lX79+ANTW\n1vLNN9+kPrdq1SoGDBiQ9Xod0m0H9g4Ze1NCZ/PH7CiGSsDyzrVIo7y8PO/1dbRfFtkmEgkCgUDe\nKgtrv7sD9tdua18DgUCb/OfWkmmtW7eOt99+m/+99SYfvPEqH3yylHBLhF2q/GxsDLGm2WCsgD9E\nm9lJQrkAYrbBlIIt7s4b4zDHgDFS8CCafjme5/8CNwpBAJipNcOyabaJxczFhmeBR4VgAHBsJILx\n6ac0f/opTcC7QhD2+wl5PDQCDYkE9ZEIpYEA/Xr3pn///tQOHszg4cMZWFtLIpFg//33p7a2ltLS\n0pyOBzJHuvmmF6x73sKkSZNYsGABM2fOZMGCBZxwwgmpn8+dO5epU6fy+uuvU1lZmXVqARzS3QId\nkW0+88fyISA72VpTIVwuFw0NDTmvqzMUQraZcrtbO5oE2qQcLKTni+0KgPaIOJdjWblyJf/973/5\n77P/4bVXXqGhsYm9+3jZyxfiJ2WKMbuZnzvx3QTrQ4qbPTBNQjabeEvB2YakRWmuQ7NPjp2Jm4Gr\npGSZUpwMHKp1XrKlNcBcKalXislaszvms6ENtIaWFnNJQgGhUIiGUIjGlSupf+MNngI+xCSgXqWl\nbIpE8Ho89O/bl9raWoYOG8YOI0cyaNAgBg0axJAhQ+jXr98Wv5N00rXmxeWK0047jRdeeIFNmzYx\nePBgrr32Wq688komT57M/PnzGTx4MI888ggAxxxzDIsWLWLEiBGUlJRwzz335LQt0QkpfPf1O0lY\nN6TdtAPazh+zpE/Z3oxNTU34fL6sdILWPtjJ1i610lpTV1dHVVVV7geXhkQiQXNzc2rkjt/vx+/3\n50WYmzdvprKykng8noqgQ6FQXlFLIbAsBq3ZbNkiXRlgLelknD7kMRqNcu+993L/PXfRXLeZpsZG\nDu7j4uBgiO/1glElIJOnUym49DO4+xs4wS25QSqq0k61Uubr/Bpgg4ZNGjYouEPBKqAvsJ+1z7Te\nmCK5uDAJzAP4gADgx4xu3wSGAqcDtcmf54IEMB94G9hXSo5SKud12Nf1IPAJsLeUTFQKb/J4WoAG\noD75Z5PbTSgQoEEINsViRBIJ+vfpw+DBg+ldXc2BEycyffr0VCpMa80xxxzDyy+/vC08/Nvdge0+\n0rVuung8TlNTE2VlZUBx5o9lm15IJ9t8R/BkA7ukzTq2beAC3WroSKZluWdZD+NQKMTzzz/Pvx77\nGwv/vYSAhLNq4bwhsHtZK8na8cxGOOU9aDTgIAHrDcVJStIkBS1K06I1EaWJYhJniYCAEHyrNH5M\nsh0rBAEpqU9+RtIqsFfJxUgu8eSywjAIJT9fCWwE/pj8Pytz4RbCJGohzAXwGQZBoCL5vTrg9eS/\nLwIG5un/AfA/4HEh6AX8SGv62dYlgGBy6W/9MJGApqbUZ2JA3dq1/HftWl7GfNCeddZZGWsp2zK2\nW9LNFNlasq9C549Z6Ix08yHbfCU6FtnG4/GUEiHXqHB7ghACt9uNUopXXnmFhx6Yz5NP/puAx6Ch\nPs6oUvjrbjAumWKPJODpzbBkI7zXDN9EBavC5mt8ABjtFpS4JMOEYjCKGgE1EvoI6C3MYplXwGcJ\nOCUqCCnNFcChgNAasvTKaAauEoJm4PtCcJzWbW5yjUleLUBYa8LJP5uT3w0BjVKyRmteTaYhfJjR\n5y2AF/BKiVcIvFrjV4pKoArzATEA6EdbYqkH7pGSDUpxlNaMoYMwsJNjez4YRFdXs+TOO9l7772J\nRqNFkTB2J7Y70s3k+AWk7A4TiURB88cybS/Tz3Il23yf3tbgQrt1JJiRfDFgdd/ZvRe+C9i4cSP3\n37eAe+f/mYAMc+r4ZpbXCN79SvObESbZ3vkNvN4kWRPV1EU1fX2CPSolzcpgQ1gzwQ13BGCoBJPu\n2ifOhILpLfBoHI6TMA8oyXGf7wfuEYIdheBmrembSdqESaI+zEg2HQq4Xyk+BcZJyUlKYSWKokAT\n0KQUTZgpgEag3uViBfC+UjRqbcrYAK8QhJL7oJRiz+T212MSdLZ55QTwutvNmx4PMy6/nAsvvjjj\nm2ckEklZTG7L2G5Itz2ybWlpSY3EEUIQDAaL5paVKelfSBrBipyzITZLPpXJpzdfi8h02A1v7M5P\nYJ7Xbdn8uz188MEHzP3TTSxevJgT9hQ8OD3CJ6vhgvkQU5pBAcF1X2ncAvbv7eK0gQb79IKxFbA2\nojnhTc36ENwThBPd2RXKHo3BhVFBLw13oNkpx9/Pp8AvpaRZKX6iNePzjPg+AG6XEpfW/DiDusEi\n6z7pX0yLwuPAe8DDWlMO7IkZPdcJwddK0aQ1CUxS9guBTwiChkEVUA0MBGowUyMrgKeCQUbusQfP\n3XorQ4cOTT3orWvOuq4bGhq6XKNbDHznSTcbsrVadevr64u6bTsRFSNnm02OuCOyzWU9HcHeEQdQ\nUlLS5v9CoRAulyulArH0zem62W1BOwvmPr/00kvM/sN1fLb0I356SJTZv1Pc8xIc/FvzJvEKOGqA\ni+P7GhzUG4aXgBDmcSkF530AD30DZ/jghrKkBKwTrFFwcotgaULzUzTHYBJWPW1jYqtI5k0uFiLA\nb4BXgaOAkzFJMVc0A7OF4AutOVprDtKafMOOCDBfCL7UmglCsJ+V3kgj8BjQoDX1WtMA1AlBnZT8\nT2ueU4pI8nOVpaXcctttTJo0qU39xa6Z11rz2muvsWHDhoILuHPmzOGuu+4C4Ec/+hEXX3xxh8Y3\n+eA7q17IRLZWR5c1f8zv97eJahsaGlKja4oBK4dqGAZut7uNGiEf1NfXU1ZWljESV0q1ObZAINCu\nT2++Soj09uNAIEAoFKKsrCxl9APQ3NzcpkCXLteyL5Z2ttCoOB/1gtaaq6++mv++8BShutXMPCpM\nOAbzX5K8+7Ui6IKj+8HlI2CvysyFskVr4Zz3JL2U5s9eMwL+yIBlClYqWAc0CEkYQRSIak1MKTYp\nM9eboLUQJjBfuS1FQmo/aS2YgUnCBq0qBIEZNbqtBXBrjTepMijFbKywcq+9MSPJKuBfwGNCMFwI\npihFIW0FLwL/FoIBQnB8nuvSmFKypzwedh87lvseeICampqMn7UCDLfbzW233cbChQv55JNP6Nu3\nL7vuuitXXXUVEydOzHrbH3/8MdOmTeOtt97C7XZz9NFHM2/ePO6880569+7NFVdcwY033khdXR2z\nZs3qbHXbj3rBsldMNw63or9M88csFLODzIpsgaLliDPtXzrZ5jLuJ9tUhd3KEdhiplpn5ywX05t0\nuVZXjWdZtGgRZ/7gVAJeOO9geCMhuPSv5k0fjyqqfXD/WDi8X9vvxRQ8vxGeWgd3rjADOK9QhICD\nw1AmoL9bMsAlqPXAfhj0E6ZErJc0C2W/DZukdy2wC2b06oGsosuvgUukZJ1SnAPsDCkyj2hNhGSR\nLPlnSAjz1R5YoTVNQLNSNCS3CeDSmm+FYJ7LhU8pSrSmEpOc+2KqCfrRfg52HXC3lDQqxSSt2Vnr\nvAplm4Gng0GoruZfd93FXnvt1eHnrWvC7XYzY8YMdt99d95//32mT5/ORx99RG1tbU7bX7p0Kfvu\nu2+qe27ChAk89thjPPHEE7zwwguAaXxz0EEHZUO67eI7Q7oW2VomNOlkm+1InGJ0kFlPX5/Ph1Kq\nS+Rf6WSbi5tZtuRlnVNrEnFnTSG5KiuykWsV2/Rm+fLl7H/A/hjREG4XBHyCuUs0p00Arxde+QjO\n30Fyw04KF/DQKnhyHXzQLFkf1WyOahLalDbt74FTSmC4C3aQMMhlpiFaY9JWNCs4pVHwYkxzrhCc\nhSYXEaICrgf+DUzAJOysCm1pyocYMAd4BzhICL6XVDA0K0UjZqGsUQg2S8knWtOgdWsOFvBJiU8I\n/EpRqjXrMeVoI5TijGz3KQ0J4HWXize93g4LZZkPr/Was1qAhw4dytChQ3Pej1133ZWrr76auro6\nfD4fixYtYvz48W0cxGpqatiwYUPO67ajx5NuJrJVSuU0f8xCvqSbTrZWZGsfAlkMWMfW0tJCJBJp\nk48uNjpyF0vfJ+vPYpr7pD+oOjO9sc5NIpFo0xZsIRQKcdUvruCv991PQkHQC0P6Ca44UdO/F5x9\ni8CT0PywFt5rUIx8QbIxougfEOzfT3LBCINyN1z7kaSxRXFXGUzKMoE6Jwy/CglGCcGjaGpzPE8v\nANcKQQlwndaMyLMQ+hxwrxD0FYKZStE/uR+90z+YQaIWJ9m4oBT1mJrbTzBJtg+wGriJNGI2DCpp\njZYHYj6s7FiJGd2OHDeOV26/nSFDhuR0TOmkW4jD2KhRo5g5cyaHHXYYZWVljBkzpksCph5Lup2R\nbS7zxyzkShztkW2+6+tsW0opQqFQUci2PSWE3fCmI7LtbnRmehOPxwEyziZ76qmnOOOMM3BJ8Ljh\nsDGSa05R7LmD5sBfwltfQplbEVXwUoPgmP4ws59ivz5Q6dUoZXDum/D3r+HsIPy+CkqzuKyWJuDE\nJsn6uOLXaA7N8bW7DrhECD7VmtOBI/MscK0HbkymJCZrzT55vP57MMnVDywUgm+05kgh2Fu3thQn\nMCNli5jrgDqXi2Va85ZSqWYNv5Sp5gzKyvjjLbdw0kkn5Xydpd9bjY2NOZN2OqZPn8706dMB+OUv\nf8mgQYPaNb7JFz2OdC2ybW5uxuVy4fV6U8L/QuaPQe4dZO2Rba7r62xbVsMGmK/4XaFFNAyDcDhc\nFMObXKRthcJKUViRcCAQSP19zZo1nH7GZD5492P8HjhxH7j2VPh8jeIHt8C6TWae8tQhgpNrNYfW\nQC9v29/X4jVw9muSKq15uVIzztMaZSplKhCWKVhpmH/foGCTAYviZjsvKEZh5jz/AighUm28Ivmn\nVTyTSQKTWrNaa+oAtGYs8LXWPIAZWZZhFsWswlgvMt/ICrgLeB4YD1zIlpFmLlgEPAMMF4JLtaYs\n7dp2J/elTQHNFjErTFJ+SSneAHYcPpznXnihYJmXdZ01NjYW7DC2YcMG+vbty8qVK3nsscd47bXX\nWL58ecr45t57700Z3+SLHke61qulVXCxxpcUMn/MQrYdZJ2RbTGQaVuRSKRoRGYdq71TrRjncFvB\n3fPvZsZlMwh4TbI962C44xnYeyaEwuBzwc1j4dxh4JJb/s4bY3Dsi/DfjbC7WzFMwgUhQR2SZqUJ\nK0VImZFbhQt6uQS9XILVccWmhPk6fTwmyfkAb9K/1mrjtRQK1pYTmIqEpcCTmOR6MhCTkogQtGBK\nycJaEwJatNlCHE1+14PZzusFPFLSkmwDVsBwoEwp3sEsiNVgqhmyxUrgLimJKsWpwI55pjfqgcXB\nIEa/fjxz553ss88+ea3HQld46Z588sls3rwZj8fDvHnzqKioYObMmUyZMmUL45t80eNIV0rZRgpW\naFRmh5WiSIc92syFbPOJdLuL2K0io0W2Xf120F1Yv349I0aMwO+BqlKYOBre+Ury7/8pqsogHoHD\nB0juGq8YkAz7EgqeXQuLvoX/1cEbG83X6IiG/hIqPJJ+HthfKga5DGpdMMAN1RJKku/W6xOaSZug\nLgE/F4IpOaYCwsBlQvCG1nxfSk5Ryiy0ZUFwBqS8btcCtxoGUeBwTGJvcLlYCXykNY3J13wB+JKN\nCVZhrEzrNg0K/YG/YUq49gcOgpyKf/b9e83l4lWvl8t+/nMu+tnPiuIHnU66heZ0AV566aUtflZV\nVcWSJUsKWq8dPY50Y7FYSnwvpSzqq3Y6geRLtu2tryOkN1Bk2lYxCM5u9A3kJDHb1nHdddcxd+5c\n3BKiCTN/+/FqwTlHKR56TrBmnea+/WCf3or5X8F/1sHysGRDWFHuhdpSwdKNmnIJd1bB8QHwtKNG\nsOOaOpjdBAdIwWI0fXL8HT0I3CIEQ4XgT1rTP8dI0oUZuT4FPAqMlJLTlCKlwk4riqUcvZLqBMvV\na7OUbEo2NqxKun8pTJL4WAiWSUmpUvTGJOZaOm/nXQksCgYZvscevHT77QwbNiynY8sFhXjpdid6\nHOl6PB4qKipS86CKCXsHWSFkm76+jmBvOMimWy1f0k2XmFkPrO8C4a5du5ZRO480O7dc5nL0OBdX\nTjF45l3NDQ+YBbFhpfCT/wnqo5rde0sO66+Z2VexVx/49Xtw/+eaM8slfyhXlGdxWt6IwpRNkqih\n+ROa/XIky+XARUkjmJ9ozQF56luXAX+UkrBS/AjYrZP9yOjoBaAUm4F5UuLDTI/shJkWqDcMsziW\nJOYvlKIhKSXzCbOd1w+UKkUfzBTGt14vX/j93PCHPzBp0iQAwuFwwd7FFtIj3ebm5pRL4LaMHke6\n9l9SV7zWGoZBfX09Ho+naK/2mYpK6d1d2dhH5nthWg8Qu+qhoaGhKOdPCJHqBgPaFLW6A2eccQYL\nFy7E6wYp4fRDJL84VbFircEBl0GJG5SGcq/gsFo4sr9mv77gd5vE9NwaGLtQUmponuun2dvXOXFG\nFEzdBP9pgTNRnIWpf/0m+WectrGxxHwt92NKqvzADcDTwGHA78ivwBUD/g/T5/ZQ4FjyawMmub//\nxCy67QqcSavmthwYnPpg2/MTBeq0NhdgsxC8nPzdH/G97/HO/Pmp4laxm2HS7yutdZfIJ4uNHke6\nFootx7IrBIrZQZZpW/lO8s0nXdHS0pJRYlaMHLgl2UskEqmR7NbbR0tLS9FafDNh3bp1jBy1Iy4g\n4IUfHg4/OwHuXKwYfhYEXHBAPzhzBBxXC/2Dbc9bOAGnPC94cY3mmkrNjFJNSMNzLfBhHJYlYFUC\n1mtoFC5atNn1tSqmCNLauvuX5GIvkqUXyuxtvBGS+eLk559RiucAl9W+a2vl9WqNR5nbK8f0tK3C\nlG59AzwB1AjB1XmkJOxYBtwpJVprpussx/gk4cOMamswO8r+HQgwvHdvbr/7bvbdd982ny20GaYj\nMt6W6gqq6eJPAAAgAElEQVQdQQhR2+NIt5hi/PQosKSkhHA4XNTClX0/O2qlLRayyQ0XCrviwfLl\nlVKmbpRwOJxKYVg3U3pUU8jQyKuuuoq5c+cS9MFx+8A+o+C+ZyV3/Nv0S9ihFP42Efbuu+V3Iwn4\n2Zvwl88BNIM9gjnNgt82aCIKqlwwwCMZ5BHUBjTjpaKvyyCq4PrNplzrN8CBtE5nyOaRuRE4T0o+\nVYpzhODY5Kt5jOQwyKQSIay12b5rW5qEoEFK1gLvKMVarVNkvlJrfk+rx60PCBhGiqCrMT1uB7Bl\nNB0F/iIES7XmIOAgrfOKwgzgFZeLl71eLr7kEi657LKsp6VA580wFhmnT4C27qtwOJxq3d2WlTdC\niP2A03sc6QKpJ2C+FoXpZGsRk0UKxYa9My2X+WrpaE9dAVtG0J3lhvN5aKU7mFVWVtLc3Nxup1p7\ns8o6imo6iopjsRjVNdW4hUFZAEYPgec/EDz3vsZQioALZuwmuXo3hddlRrP//Br+vQo+aHSxqtkg\nEjNbdQ8KwPiAYGe3ZoRXs4PbVCS40gpnSsElG+GuBjhOSn6JIldR0h+Be4C9gAVAVfK8uzFJu1P5\nltYow2Ae8D7myJypSlGCSdhNQGOyjbchudS5XKwBPtaahqRiwUMy/yol4aSkDK05EBiiFDFyf/X9\nBniipIRBu+zCi3/5C8OHD89xDZmRjV9HPB5Ha828efOYPXs2ZWVlnH/++ey2225MnDiR3XbbLeft\nzp49m7vvvhspJbvtthv33HMPa9as4dRTT6Wuro5x48Zx//335xvI3AJc0SNdxmKxGIZh5DxuOZ1s\nA4FAm1edfN232ttWPB6nubkZKSXBYDBvsrUQjUaJx+Nt7Ous7VgFCovUO0Mu89s6cjBrbm7G4/Fs\nEen6fL6s82vtOZDZo+I///nPXPOra5KkCF4PjN9ZMmGM4q5/CqrdcMe+mjc2wuMr4YtmyfqwYmCp\n4IABko83GHy2GU4ul8zprajKYteeC8EZ6yUeQ3Oz1uyZ1dG04n/AxVIS15rLk00O+eAd4A9S4taa\nc7RmxzzWYTUmfAncnfz3XkCTy8Umram3d4wJgU9KAkpRoTX9MCPlIbQ+IKLAM14vH/t8/P6mmzj1\n1FO7Pcq0pkZ4vV6WL1/ORRddxOTJk/noo48YP3485557bk7rW7NmDQceeCCffvopXq+XqVOnpoZQ\nnnLKKUyePJkf//jHjBkzhvPPP7+z1bU5GUIIN/Cm1npcj4x0Iff8ZqZiUkefz/cCSnfkyjVvm+92\nciX1bJUVuZy3fNFRri8ajbLDsB2IhJvxuMAl4ZRDJJeeqvjVHZobFkBfn6ZeSyY+pRleKThmKFw+\nQLF/f1hWr5m8WKOigsUDNBODnb8dNSs46VvBK2HNRUJzrs7NnCYMXCQEr2jNFOBUrcn+ZbsVIUzP\nhU+05gStOTzP138LT2AOqhwnJccpRQAydozVaU2dYVAHbJKSFULwTtJ8XGIWCkvdbo4+9ljuvflm\n+vTZwta8W2A9lMG8zwYMGMCFF15Y0DoNwyAUCiGlpKWlhQEDBvD888/z0EMPAabL2G9+85tsSDcd\nXuBtIcSxPZJ00yuWHble5UIahTyprfbkdJOYJttgvUJhEaXdjKYYEXQ60hs08iHbYqRpFi1axLRp\n0/C4TRnYmccKfnmWZu4/FOPPglKPpsQNBw+RfH8HxeGDoMpvbjeh4Af/gX99CZdUaa6p1vjbkYEl\nFKxOwIoE/LkBHmoG0JyIaYd4OaQ6vyxlQro6wTIaXwmY6WLNRCChFI9hetpa7bu9ME1mOlIsPAI8\nIEyf2xu0pncB5/NjzI4yj9ZcoDVD2klRScxiXQXm9GCgjVqhDnjA6+XrWIwbbr6ZKVOmbNU5e8U0\nuwEYMGAAM2bMYPDgwQSDQY444gjGjRtHZWVlitxra2tZs2ZNPvsaFkLcBtzSI0kXWnM+7cmx8o3Q\n8vEN6MiRq5gqC6sTLxQKFWxGk2m/0vPC2RThMq2n0AeAUoqx48ay8uvl+L0w7QiYfizc9KBm2Ilm\nC+/0neEHO8H3BoBbtiWRx76E858T1Ap4bqBmk9L8oQ4+jcEKA+pwma28hqZFmQU0hVmJ9wCjgEqX\niybMIlEAqEhGmR6tcSf1tJY6QQnBt1rzZHIdhwKGy0UMc5ROyudWt3reRmmVklnTeL1CkFCK+mSR\nbaDWDNSa9yD1it+L7GeLhYG5QrBMa47Qmgk6P8McBbwiJc95vfz4oov4+RVX4PP52s3ndxfSSbfQ\nFuD6+noWLlzIihUrqKioYPLkySxevHiLz+VZj6kFRmutD+mRpNseoRXjdTgXkszG/rAYpGtF0NZ0\nhoqKiqJHtt2hrMgGH374IQcccABBPxyyJ4wZCY+/LLnnSUWJGyYMhPsOg4FpU1kaY/C3z+Dnr0JL\nHIJS86WAA1dBX49gqF8w0i84ym1Q6zEY4IEaD/R1wTVr4aHNMFlKrlGKMsh6+q4CZmrNYuAoKTlP\nJQc5dvJ9jUm8zZjtu/VacytmYeoAzNxpvcvFauATbbbvNtP6cPBKaeZetaYk2X1mte8OxswDPy4E\ng4Tgcq1TxbtcsQp4PBik30478fxdd7HjjvlklLsexSDdJUuWMGzYsFRN56STTuLVV1+lvr4epRRS\nSlatWsWAAQOyXqcQQmiTAAYCk4GHeiTpWsjUQVZo7jEbkrTbH/r9fnw+X5cQlF2aZbmLhcPhom3L\nSomEw+GCUxXF2KcjjzySt996Db8XhtTAW5/C+19CY6PZpnv7QTBtR9Pu9YVV8ODn8OYGyZqQZkNY\nExQwNgjH9YHRfhjlh6Fe8AhLYNUWixvgyG8k5Ybmn2jG5qiGeQ6zUBbUmtlaMyqH7wtM5YIf+Aiz\nDbgSuFbr1kaEDMQdpa2vbT1mQ8ImKflQaxYrRRzzaN1as0EI5rtcqcGP/TDv/iG0jvvJhCiwxOPh\nXa+Xq6+9ltNOPx2Xy0UsFtumZtvZHcYKTS8MHjyY119/nUgkgs/n49lnn2WvvfZi06ZNPPLII0yd\nOrUQlzENlAshftqjSRfMscvxeLxohZ6OSDedbLMx2skn0k13/rLMaCzv4GLAyttGo9Gt7pu7du1a\nRo4aiVeCocz8bd/esNdu8Ph/4Kghkgt3VfztC7jubcnqJoXbBQcOcnHEDgb3fgC93XD3IDghi2Cn\nMQEnrhC80aS5XGh+lGOBqg6YLgTva83ZWnNSnq/tm4GrpWS5UkzTmoPpPHXgwyTONo6uSUnZfMzp\nuftLyeFKEQbqbN62G10uPtSal2wSMn8yYrZIuQazE+35QIAJhx/O27Nn06dPn4ydZLDl1OfuJOR0\n0s11PE869t57b0455RTGjh2Lx+Nh7NixnHfeeRxzzDGceuqpXHPNNYwdO5Zzzjknn9WXYNpVXNgj\nJWPxeJxQKJQq9JSUlBStqp5JSmV/vQ8EAjlFtlZkmk3BIV0Hm+6NYBgGTU1NBT3RLUKPxWIpX4lC\nbhLr+DweD4ZhIIRIdcFlo2X8yU9+woN/fQCRNBg/dgL8eArMuFHw/jJN/yDEtSCc0Bw8VHL8MMUh\ng2F4Jcx4Du58F07rLbmpRlGexSVwy3r41beCvYTgj0oxMMfjnQP8CdhDSi5Rigz9F1nhTsy227FS\n8gOVu/bXjvcwO8oCWvMDe6TcARRmxLwJk/w3AV9KyedK4Xa5ePSf/+Tggw9u//vJgQF+v7/Lho12\nBK3NqdNWQHLjjTdywAEHcPTRRxdtGwUidbBCCKm1Tr0G9chI19LpejwePB5PUWVM9si0GF6z2US6\nhQyXzBaZCN3av2LAanjIdr83b97MiJHDUQkDrxcmHQQ3XAxX3AxHnQ8+l6amBKbuIvn+jor9BrYW\nzF5bDYc+JPHENc8M1+xXsuVrfX0CPo7AshisiMH7YVjYaBbGeqNZLwSnuVwksE3ktXV62SGAkDLH\ngnuS/14lBDNdLtxa41bK7ATDVCSUYioALJVCX8x8awWmX+71UpLQmsu0ZpcC2ndDmGmJL7XmuGSh\nLNurRtJqOK6A14Rgvc/HJeeey1VXX51VkNBRJ1l7nYjpHguFXn/2SLfQnG5XQWuthBBB4Ejgez2S\ndK1R5uFwuOgdZEIIDMOgubm5YK/ZzmDPRWdDtvl2kWUidLu9YyGwOsus101rndbNl+mV84477uDy\nyy8n6IcJ+8GFp8FNC2C3SeB3w8V7whmjYY9+IEQrKcUSMO1f8NQy+HEfxV4BeKwerl8La5Sk3oBm\nQxNKaOIaKtxQ5YZlETN/eZSA4cIkvyCKEm0SpRfztd2dXKRofc2PKrhWm9HgScBYTJKOGUZrCy/J\nCbxS0iQEmzBf85u0JqQ1zdo0Hie5LakUASm5R0p8WptNCJgEbakUBmGSd3t4EnNszjAh+KXW9Mrz\nd7kGeLSkhLKhQ3nq7rvZZZddsvpeewqfzjrJrGvFGkSQ7q+Q7bDRTAbm26Ktoy3KPQ3Tl35JjyRd\nC8WUYwFt2lIDgUBRyDZT626+hb9iNoR01FKcDeyRs2VJaaUXrAkX6W2+SilGjRqFkWhi52Fw6L6w\n+L9w3E8h4IZpu8CcQ6HMZ98OvLUWfvNfeHq5OZtMAnPWQ3+fYKegZOdSzTE+xRAfDPJCrdfM8T5V\nD9O/lOwkNbcLzR45/iofVTATQbWAO7WmUyfYds7nE8A8IagFfpjM/zYqZbbv0tq2uxL4QJttu82Y\nx5myTUzmXd2YBjVx4MRkdJvPTRwDnvF4eMPj4VfXXcfZ55zTpTafmRpg0v0V0oeNZlrs37VjWyVd\nWtMMRwG3aq0X9XjSLYQ4LBiGQSQSSRGI1SJcDNiJshhNB52hqw1v0qPz9DSFFaVY59H6zjPPPMPk\nyafgdcPAali3GeY/BioO/UvgwUlw0GDTkObeD+GRT2FpvYtVDQbKAJ+Ey/rDPqWwaxBG+MAjNWbC\noC3qE3DIUsGbjZpfuDTnCZ1qH84G3yr4AZIvlOJCNMeSvTbWjtXAL6RkrVKcrTX7Q/t+uRmMxkOY\n3WGbtWYzptl5BLMa0ywEi7XmCcwo3S8lfiCYNBm3ouUhsEU33GfAo8Ege33ve7x5663U1NTkfGyF\ndG1ayMZfQSmVcdho+n4Uozmii2A9HZYB+wghlvZI0i1W40F6nrOioiKVLy4WrAeDRbbFIML2GkLs\nHgzFNrxpzyqyszSFUoqxY8fy7erlaA0uN5SVw8AB8L934ZzdBccM08x9B6Y/Jfm2UTGgXHD4CIHb\nY7CuHg6qktw5VNE/i17aW9bAr78R7CUFr7s0A0X2x6gU/FbDXzQcKOF60oYsZrseYDbmIMcDgF/Q\n6k2bLQRmeqEUWAs8JAQVwCVWoSx5zqOYyoTNSrEJsyC2weXiPa1ZklQweGmdwOsTAlVZyZx587al\nolMbZBsVa6159tlnufLKK/H5fNxyyy3ssccejBkzhoEDcyuRfv7550ydOjV1X3z11Vdcf/31nHHG\nGUydOpUVK1YwdOhQHn744Zxyx7YC2v8Bc4E5PVK9YHVOWe5dubrFd6QSyGQqU8h+hsNhotEobrc7\nazOajrB582Z69eqVIl271hZMD4ZsGhuyPc50MrfWb8FqDvH7/cTjcaSUqSj7s88+Y//99zf3KwDj\ndoXpU+A3f4BvNsDQCmiICZTWHD1SMmmk4tBhZnPDcQ9I1tQp7h4GJ2bhP/RZC5zwmWRjVHGLgGNy\nDE3fUPAjzALXL7VmXG5fT+FN4AYp8WrNT3R+5jQWmoGbhGC51nxfCA7KoVBmIYFJyk8ArwPjdt+d\nJ596quDrOx6Pp6STWwPW9oUQLF26lMsvv5wDDzyQDz/8kKqqqpRXQj5QSlFbW8sbb7zBbbfdRu/e\nvbniiiu48cYbqaurY9asWdmsJt3wxqe1jgohxvRI0oVW0mhpaaG8PLvZpp1JsoC8idwOe4eXdX6L\n1UVWV1eXKoYV0tiQzXFaZAvtW1JGIpHUzWcn3TPPOpPFixZTEoTRI2H2r+BXN8MLr5rFsnI/nL6H\n4JSdNXsNNKc+AFzxNMx7HU7tK7m5VlGRFqxvjMH7LbA0DMujsCoKSxqhyTCVBX1cEgNIaEigSWgw\n0ChtRqDp6gQj+W9v8k8XpqG4ZUjuFgKXEHgg1QbsTdoqWsbivTF9Ff4BfA1Mw5ziUEji6DFMotxJ\nSk5XinxfnNcCfwsGEQMHMu/uuxkzZkwBe9WKbYV0/X4/WmuOOeYYXn755aLcY//5z3+4/vrrefnl\nlxk1ahQvvvgi1dXVrF27loMOOohPP/00m9Wkk+4vgAe01it7ZHoB2novdIZcJFmFpCzSo85AIJCy\nOiyW+sFqkrBmxHVFR5y9CSTXxol169ax4447UhKEEUPhuhlw7yNw+FTwSPjx3jB9LIztD8L26v/O\nGjj5IYnRorh1CDQaiouWw+cxwUZD0phQNCVMEq3yQI1P0pRQrGuBAQIu95pmMhXCbMMtFaYyISBa\njcbdIqlOwLwj7ovD9XEYLgRXak0lZoEqpjVxzPxpVJum4tZiGYs3CtNY/FvgEcNIqR9cmBN0H8V8\nlfeK5MRdrSlTil6YCoX+mO261bTVba4AZsvs5521hwTwtNvN8x4Pl8yYwXkXXIDX6009GAuVaxUj\np1sIMm2/WPvz97//ndNOOw0wr+fq6moAampq2LBhQ76rPQe4EXqoTtdCZ4W0fPWv+ZBuez4MhmEU\n1fBGKUUoFMpbN9zZ+u1tx7mu/9prr+X2ebOpKIcpx8Lr78JpF5nKhAv3gesPBZ/tilMKnvkSzl0I\naxsh6FLENFz4NYwokexaIjih3GC432AHPwzxQ18PhAw48RPNsia4xif4qUvjzuE0LFcwJS5ZmVBc\nCRyp8xgIqTVfGgaXS0kF8FPAGk4TJ6lK0GnTdoVgo5R8oDUvJv8vhlkI8wlBU/I6UUoxATM/+xUm\nOedyo34BPBQMMmqvvXh13jxqa2vbNY7P1E22LbT4dgY76cZisZwmVXSEeDzOE088wY033ggUdRLF\nJswa6IoeTbr2kR12pJNtri5juaCzqLAYsjZ7WkQIQWlpacG5Yft+2c+XVVDMVj5kRd6VvSrxuOL0\n7Q0JA+58CIIeGFEFD081I9tYAu55B/7xCXyy0cXXmw1KJAzywW92gHElsHspDPC21efacdsquHq5\nYLxL8G5AM0jmVij7eRzuTcAR0qxq5JNESgDXAUuAI4AfYEbVFjyYKYfe6V/UeguVQhxzQOW9WtMf\n2B/Tw3ajEHyhTXPxCK0KhYAQlBgGfTA9FIYmFx9mBP64z8cngQA3/elPnHDCCalrMVMTQ3tDItOJ\nOFNUvLUjXTuKYXZjYfHixey5554pj+Dq6upUtLt27Vr69evXyRq2hBDChenW+WshxMM9lnTtv3Dr\nAuhOlzHDMAiHw6nW4GJHnWAeV0tLS5tIvampqaiGNy0tLVk3Z2TC3//+d376k/OIJ8BXApE4lATM\n6PbS/WDSKJj9KryySrK6wVQmTBwO7601CEr47TDBRQM6l3Qtb4HjP5Z8G1bc7tWc4NLkchqeT8DZ\ncbMZ4XZ03q/tLwC/TSoJbtS5DXFMRyNwQ7JQNsVeKEtbZ5ykd0JSobARWO9y8bbWPJ3U9GpMUp42\neTL3//73nZJQJoWAuelWIrZHxekkXOympFxhNzBvaGjIuq7TGR566CGmTZuW+vekSZNYsGABM2fO\nzNvsRmttCCFuAmYAF/fYQpp1MdTV1VFeXk4sFmt3DE8uUEp1OAYovTXY7/d3SIJa5z4CKP3hYT+e\nxsbGglUQdrL1eDwEg8Gcz1cikWDHkTvy7ZpvKSmBygqYNhnuvw8am2HHKljdZCoRDh8lOXlXxeEj\n4elP4dLHBHuUCBaMVAzppA6jFFz8JdzzLUz1Sn7nUlRkON1KkRr0aLcFalFwdgJeN+B8IThN5zYF\nwkIdMEMIPtOas4TgqDyUBHb8DdN7YdcCvRc2AvcFAnxuGPzp1lvbEEaxkB4VW4QMZBUVdwXs/h5v\nv/02jz/+OHPmzCl4nYMHD+arr75KFZg3b97MlClT+Oabbxg8eDCPPPJItnpgIYTwASOBr7XWTckf\npteGew7sEWljY2PRXcbSX58yaXqziQrt0q5sWhstE/H29LyFFvrSRwnlo9J47bXXOOSQgwkEoE8f\n+N1v4JFH4aY5ZjeZR8Keg+H/xsGE4eBxKdY3wXF3ST5fq7hthOb0fltGqmsi8E5zUpkQgdcb4f2Q\n+epcKmCJAYsSkpjWJLSlTGhdBK1NDAKTfAOYBTCA27RmHq2j0t0kR5+L1vHnHsCnNf6kL24FsBz4\nBOinNTOBnQsg3K+AG6WkRSl+Qv6FMgN4xuXiKa+Xiy+7jEsuu6zoI6EsZIqKw+FwSs3S3rTe9Nbe\nYsJ+DxQrvRAIBLYolFVVVbFkyZJ8V7kjMBP4EYAQ4jDg3h5LutFolKamJrTWlJSUpEYwF4r0i6OQ\nfKd9nR2RbroWtitmqtklbNZgyebm5pzXc8QRR/D66y8RDMKVM+CTpXDJpaA0nLw7nLcvHDLCtGe0\n8Ptn4Xf/gaOqYN5u8GYTnPUZfNZiKhMa4qYyQWvo6xP09cEHDRofcLEfdnJBlTCVCRUCygSUCAgm\nlQk+YeZRZfL0fpGASSHJ2oTiGsxJDprWcTsx2o4+b9GaMOakhdTYc8wxN0swSX8HICwENyXVDDL5\nc69IDnHUmmCaOmEIZhEsmNz2bOCN5P6ckPx+PvgKeCAYpP/OO/P8nXcyYsSIPNeUP6yCW2e54kzd\nZMWKiu1mN9toN9pAoEJr3SKE2AEz/T+/x5KuFaVZRFVMWKoDS8vaVc5fmRobOtPa5hrpWrnn9EJf\nrl13GzdupLa2loAfjjwMQiH41XXg98DMg+HSiab21o7HPoQp95mv/qVueGqT4smNMLxEsmu5YFKN\nwY4lBsOCMDRgSsEWrNLM+ERwgFeyIKgYlsOLi1JwXggejMEJEi6mtRNMYGpxvXTeHRbD7CJ7HThB\nSqYlXcSsLjCrRbcBqNeahuQQxzpgo5QsF4K3lWJzksytaQ8Ss3C3Uin+Risxd2YobqEFWOj18rbP\nxw1//CPHH398UZp48kFHhjfZdJNlch7LJSou9qieLkIvSHkdjcAk4Z47I83r9abG1xQzqW+tq6mp\nqUuN0e2NDV1hIm5Ph2SSl+Vy3i666CL++sCdVPUyW3j/tQiCXti5Gh75IexSYxLeC8vgnjfh9W8E\nn6/XBCSML4eD+8De5TCmHIYEMisTVrXAnm9KvmpSzC3RnObNrVC2OAZnhSSlWjOf/C0TFwGzhKAG\nmKM1gzOsx96iu0Wzqe3z9cB1yULZVKCKpHetEKyTkle15slkIcxSJwST6oS+mP4JQ4HhmJaQDwUC\nHHr00bx988307t07rzeVrYFCPBbacx5LNzDPZYRON2IZEBFC/Bg4EPhWa/1ejyVdC8UiXbu3AJhR\nZzFTFnaPXrvqIdfGhs6OtxjpEAvhcJi+/fpiJAy8PgiFobEJAh648hA4bx+Y+6oZ0a6oM7vKhvcT\nrNqkGRKAh8bAflmYF8z8FOYthxP9sKQSqnLY3XoFJzYL3oxrLhSaaTq/KQ7rgZ9JyddKcZ7WHE4H\n5jRZ4EHMDrXdheD/tG5bKEuTjhm0qhM2AhuAtS4X72rNY0phAH0qKljw178yYcKE5Cq2vnqgGIY3\n+TqP2fdhW00vaK3fFkL8A7gMcyrTbOjBzRF205tCnMYyuXIVO2Vhvc5bo4UKbWzIdMPl6s3bGf7y\nl78wY8bFuCR4A7Dv9yTvv6Yoc8PYgXDXG3DDEtilP5yxHxw5GmY9BU+8q7lsmODqYRpfJ+z3Vh1M\nflei45p/l2kmeDr+PSplvvpbmN0CN0RgnBQsRFOdJxHNwSTJfYFroKApDl9iei9EleJnZFcocwF9\nkosFZRg8JwQb/X5OP/NMLv35zykrKyMajbbJhW4rWtliIZuo2EqNbdy4kUMPPZTq6mo2btxIXV0d\ne+yxByNGjMj5vDQ0NHDuuefy0UcfIaVk/vz5jBw5siCzm+R+P4lpf9x6jD1VMmYJuq3iUDAYzOn7\ndqVAupFLppE9hexnQ0MDWuuUxKwQMkwf/5OueMhW/tWelE0pxQ7DdmDjhnV4vHDKaZLVKxXPP2Om\nFAwF++wA0/eD43eH3qWw+COYfo+gxg1/3V0zOoMgIqHgk2Z4rxGWNsP8b2B93HzqD/dI4giiWhPX\npgF5XGtTlWBTKEDr2HMfJgHr5M8EraoEV3KxfBPSlQm+pHdCDPhfch0nYjqCDcT0VMj1NxTDtJF6\nHThSSk5SagtLxWzxDbAgGKR8+HDm3n03o0aNQuvWaQz2pTuUApnQ3NzcZeb+ncFKnfn9fpYvX851\n113HwIEDWbVqFV9//TXvvvtuzvt11llnMXHiRKZPn04ikSAUCvG73/0uX7MbSL4oCXNHPIChtTag\nB5OuRTaW4UpJSXbmeXbZlEVe6UqB5uZmPB5PQekFe2ODECJFuIXCrkDo7Dg627900n311Vc55JBD\nCARh/wmSUaMFd8w28LhgWB+48GCYPM4kWoBwDL7/Z8HLn2l+u5Pgx4M0r9TBC5vg3UZYGZPUxaEp\nrmhOQNCF+aeAYS440g/V0kwn9BJQIaHcpkwIClOZ4E2qE7SGsxvgHy0wRZozyoKYhSq7MiGaXFIT\nHZJLc3KpA+ZhEv4uQERKGrWmKVn4gtbWXK/lU2sYVNI62WEwZr61AngJ06S8N3C+1uQ7HjEGPO7x\n8LLXy69vuIGzzjqr3Qe01uaMsEAg0CYCLGQaQy7YmqRrvTVa9/ypp57Kvffem+oiyxVNTU2MGTOG\nL7/8ss3PCzC7ARvp6jSS7bHpBQvZ5nTTNartuWblss72tpPeFWcRY7GglKKpqamgsenp+uGJEyfy\n4YdvMHgH2PcAwSMPKF59Fs45AC4+GHZJq1PMfwXOuR9AM6xEcONXgplLNRVe2Lncxe7lmmNKFcNL\nYaj6+YwAACAASURBVIcglLhhypuSDzcr/lgBPwq0SryywRMtcE6DoErDQ2mFMiuyzeYR+Q9gjhDs\nIgQzlDIJMi0F0EKrMqHOMMwx55jKhLVC8KHW1ClFHUnnMfNEknC5uDdZBKvBJOZhmJFzZ/gIuC8Y\nZPyECbzxpz91aixu/d7yyYmmD4zMFVs7nwxt0yrNzc0FqRe++uor+vTpw/Tp03n//fcZP348t9xy\nS1HMbrTWWggxEfhAa10H2wnp2s1oukKWBR03NhSr2GfJ2AzDSBX6CiXz9evXM3z4EFwu6F8Lq1bC\nkw9rdukPj5wHO/c3P1cfhvmvwt/fhndXmD8b6Ifv9ZPs30sxrlKzWzmUeyB9msPvPzWXw/zwaD+o\nyaHStVnBCXWCd6KaGQJOy7NQtga4QEq+UYoLteZQ3b7JTSC5bEF7NnK+D7OZfg8hOF5rGoHNhmEW\nwlwu3tGa/yhFI+ZNFhCCoJSUKEVvrVO+CTXAk4EAy0pKmHP77Rx55JF5HF0rOsuJGoZRNNObrZVP\nTi/iGYZR0FCARCLBO++8w9y5cxk/fjyXXnops2bNKkah0K21TmBmni4A3hJCeHos6doLCe0RWr6y\nrFxIMtvGhkJI1y7/crvdSCmLkqr4+c9/zj8fe5BEAjxeWPGlqbu96ii48kh4aRn84nF4cwVsbDYN\na4IuOKoG7hgD/QNgvtxnxscN8P03JA0tir9XwtH+3Aqes5rghmbYRwiezrNQpoDfAQ8DE5N/z98p\n2dQAXZ/sKLsM2D3TPtmUCdao841as8EwTGWClHwsBP9Mfu68adP4+w03ZJ0iywftRcXppjeWMXhH\njQxb2+zGvv1iBDO1tbUMGjSI8ePHA3DyyScza9asYpjd7CWE2A/z2XpUskHC6LGkC+176lrOX4XI\nsrJRRNhNvjuKoAtRKVh5YUv+FY/Hicfjea3PQigUol91X4yEIlgCVX0F0QbNqBqYvj888QHcvMTs\nMjt2LFx1Ity6WLCxTnPPOJjUiSRSKfjRu/C3lXBuKfyuL5TkUJn6KAYnNUjq44r/Aw7S+alT3gRm\nSInUmllaM7oAlUsM+APwGnA05ljXbApl9lHn1hSJNUqxIBhk1wKMxbtLspWpvbe9VvmtiUL2o7q6\nmkGDBvH5558zcuRInn32WUaPHs3o0aMLNbuJYva9fIHZ1Lgz4OmxhTQwfTQTiQRNTU1UVlbmbEbT\nHiKRCIlEot1un1wj6FwVFnbNcLrhTaGTLe644w4uueRnBILQp1pQVi75+D2DoAe8HnBJmLY/nHYA\n7DMCrnkY5vwbpgySzN5V8f/tnXl4FFX2/j/VnaSTkISwBiEJBGTfIiQBBQGRxVFGgUFgQFEWV3AB\nFAVEcBwVERH8uXwVUHFQGDcEAZcBJ4BCCCDLCCg7mISERbKv3X1/fzS3qDSdTnent4S8z8OjSTpV\ntypV7z33Pee8t66dXF1eGaw7C/futjw4k0IhVg+X9IFk6wPJVfTko1AgoFBAiVlQYpY+CgKTEJjN\ngqyiYgAiAgMJCwy0VB4oFkPwIEWxJLlAbb0NNZkIMRoJMxoJx9K08C8sbby3KgrjhaAejpGkLWwB\n3rycKHtUCGJcPI4R+CYggO8DA3l69mweefRRl5fFJpOJkpISp6t2XIW1TizLtmRUbK0Te5KM5SRg\nMBgwm80MGTKEbdu2VemY+/fvZ9KkSZSVldGyZUs+/PBDTCaTq2Y3oCnzVhRlgBBCNXCoEZGuNPaW\n3VdVzapW9LuuNjY4Gjk7YnhTFbRo0YLsnEzq1lMYPi6IDxaXEBZsIiQIhifBg7dCrzaWJoeDf0Cb\nJ3QUFwg23Cjo28iMWcDJAvgtD47kw9ESAydLDZwqMJOWW4pRCAL0egTF3Nb7JswxseQ2uY4GDRrQ\nsm5dwsLCCAsLo06dOuqkaDAYCAgIICAgQH1xi4uL1RfKZDJhNBopKytTq1XkhFRYWEhBQQEFBQXk\n5uaSc+kSWTk5/H7+PGeSk2kcFMSZOnV4MjeXnIICAvR66gUFEanXW3aZMBqpW1xMPbOZ+lii0QZY\nOsdCsVQ5zL3cUXaPEPTHtV2BwbID7wehobRJSODnd98lJsZV6rbA28ksbVSs0+koKSkhJCSkXCmb\ns568rkIbYcsqiqqia9eu7Nq166rvV8HsBgBFUYKAoYqi7MZCxB9Wa9KVWidYHgp3+SNYSxbWEbSz\njQ2OJvukVGHP8MaVpFxqair9+vUhyAB/+VsQ360pZfU7JXSKhqf/CiN6QOjl1L/ZDA8thRXJkFRf\ncNf1gn9lhfDU8UAOXyyiXngYba5vSeuETrRs24FeMTE0atSIFi1aqHaYsrceuCoK8pT1nyRmWb9s\nDSEEeXl5XLhwQf13/vx5zp07R9Yff3Dgjz/IPHuWc+fPk5WdjR4oKCsDIYg3GLhYVsZms2V7c9nI\n4EiMWQCsNhjYHxzMq4sXM2zYMLc13vg6kVVR0s5RT1531BS700vXQwgFBgghpiiKMgG4WK1Jt6Cg\nQP3/qjYdaCGJzVU7R2fg7OaSzib5+vXrx4FfdxLVVKG0TPD9Z6XE1IeVUyCx1ZXP5hTCD/th5Jua\n79Vrzq+dk7ihRy/u6dKFjh07qg+4dpIwmUzUq1dP3dlCp9OpE5R2mxh3ly45A0VRiIiIICIigpYt\nW9r9rBCCnJwcfv/9d/Lz88nIyCDtjz/448QJdp48SXpGBunnz6MDGhsMNAIalJTQoLSURkAjLPW8\n/wNWhoRwx9ChvPOPf5TrdvRFQ4O3oG3VlXDEZ0E+D5XJE/IZAr92GJMIAf5UFKU3FnO52dWadMPD\nw9VljbuXWyaTiZycHLe01NoiSuv9yNy5uaQQglOnTtG5S3uMZWAIgbNpgpAgmH4HPD/cot3uPAbr\ndsNXuxVOnoP4Lu1ITKzD7NnPcfPNN9tctlm7lgUEBJCbm1vhdVdk/WdduuRPRKQoCpGRkfTo0aPC\nz8jmkuPHj5Oenk5GRgYnjx7l0NGjnD59mrTMTJo2acLqZcvKHacy8nFUG/VlEsuVc1eWtNM+C5JU\nK1ohVROHMYl8YAOWXSOyhBC/VmvSdaRszBloE1hCCOrWrVtlhzHr8Vm7fzmrP1d2rWVlZTz55JOs\nWLEc3eWh603QKgpWPwYX82HCe7BuTwBmYaK4THD7HX9j67J37C7TtEY6WonFWd8LR0qX7BGRu1ca\nrkJRFOrXr09oaCjx8fFXyUHyb2T9t3WWfHy1M4M3oJUntJOzVie2NTHL7xcVFZGTk1PlSLdFixZq\nYBUYGEhqaiqXLl2qsu/C5W60POAlRVE6AkcURdHVki5XJ7DCwsLIz893C+HK8UmydZchjTWMRiM5\nOTnEtWyByVSGPgCCghT0QtCjNVwXCbf8ExQlgPziRITYC5hYt24dgwYNqvC42onI1rjdMeHZIyJt\nksa6hlS7HPU3ODuR2iIfa21UK89ImEwmr98DT0fZ9lZI2uqJu+66izNnztCwYUPy8/OJj4/n9ttv\nd7qyR6fTkZycXG6Lrvnz5zNgwADVd+GVV15xxndBjlkoilIHi63HcOCwEGJJtSZdCVdf/IoaG+QL\n7w5IQpcvTlX9ea2v1Ww2U1hYyOrVq5kyZQrBIQptuhg483sJubmWz6Ueg6JSEDRHiD7Al3Tq3JHP\nP1tJs2ZXOcKq45b3xtlKCncRsXWSprLlqByzJ7wGKoInCcieNiqfqZKSEpudZTVNJ9ZOzHKVuGnT\nJt58800uXbpESEgI//73v+ndu7fTpCvvqRZr165ly5YtANx3333069fPKdJVFEUnhDAD/YDRWApf\nOkI1LxmTcOUlt9fY4I7OG230LCMRd7j8awvTZeNEUo8kLlxMo0WbQELDFX7dVQJAcHAAxcVGCkqa\nodOFIcxtgC959tmpzJ37XIUm2DK5J2uL3eG25g7YiwjLysowGo1XeQ14o3LCm7Au3TIYDG5NUjkC\nXzdFaKsnjEYjt9xyC3/9619dPp6iKAwePBhFUXjooYeYNGmSO3wXdFgaEjsB24EDwEio5qTrirzg\naGODPKYrD5d1+ZderycnJ8fp49iCvM7s7GyOHj1K//59CAxS6NYnmLTjZRzaUwb1oqAwn+JiPRCN\nTicwmxsAm9i8eT29evWyeWwZNbuS3POlCYokFfk3leOxpwv6uzzhDNyZpPJ3WD9n7tB0t2/frhLr\noEGDaNu2bZXux+V7LzelzsTSBjwci/Fd9SZdCUcSOs42NrgSPcv2Y+v9yNwhV2iX+wDjxo1j28+b\nMBohNExhb4qgMF8PcR0g4xiUdEBRchAij6AghQ4d6rFx48lyD6itqNlgMBAZGenwQ+evL2tluqBs\nuqiuS/PKAgJHk1S2dmWobDKy1pV9ATk2d5SMSUe3Ro0aMXToUFJTU6vku3D53g8B0rDYfjyMxfrj\nqKIo31dr0pU3XqfTVbjRoquNDc6QrnX5V0XncDVy1i738/PzadfueoyX59HQMIXcholw+gAoZkg/\nCqV3AjsQ4ixhYfX46KN3GTLkDpvjkUTurv3g/BnaiFBWG7irhKu6oCqTkfznT+VqVSVdueoNCwuj\noKCAH374gblz53LnnXdW1XdhHPC1EGIf8IaiKP8BugAHqzXpStgiyKo2NjhCutbnqChCdPUB1ZJ5\naGgo8+fP55VXXgEgIBCMZVAYfw+kfgllRaAoIMYDa4CL9OjRl1WrltnctE9qoIqiuL3duDqhqpUT\nvpRV3IWKJqOK7oG8ZlnS5svJqKryQlZWFsOGDVP14bFjxzJo0CASEhIYOXIkH3zwgeq74CTMQH1F\nUZphMb3JwGJSV3PkBe2D4I69wuyRrivncEYjtq7lDQ4OplmzpuTkXGlCMBICff8OKavBWHJ5YA8D\nq4EcXnjhRZ5+eupV49JKIPIlcwfhSiMeGRFK/bA6RobOVk5Ij2NvaqSeXuLbuwfSlN8XSUvrZ6qo\nqKhKpj9xcXHs27fvqu/Xr1+/qr4LUcAALBuNGLiy61RxtSZdbSJNFu9LZy53l2bB1e5fzpzDkcjZ\nFpknJydz++23A6DXW/Yoo24z6DoYUv4NJgXMCvAIFm+tfH7++We6d+9e7thaIpcSiPStqAqMl3WO\n4uJilbyl1FNQUGBTK63uRKydpAoKCspdty9bnT0NeQ8URSEwMFB99r2ZtLQ1kftaX64AOiwbUp7B\n4qUUjIV8g6o16UrIpU9ZWZnblsrW0bO2nteVc1QWOWubMySZ33LLLezYscPyoSA9plIzuqhWmFvf\nZCHckmgsf9P7gBVERBg4ceJCudI0d0X+1tBWOoClSkMSjqwkCA4OVqPDmlrKVZE8UVmrc3WunLAm\nvsp0Yut64qpo5dpz+7m00xj4XAhxVdlStSZdIQS5ubnqH8JVj1l7x3fUqNxV2HIXS0tL4/rrr1c/\noxgCEEGNUBpEYI6Igl1fQMltwLcoSj+E+JQRI4awcuW/yo3dFpFr4UqFhpbEpVaek5NDcXGxel8k\nEUsNUEZGUgPVktK1RsTuqpzwd+nGUa3cVruzsysiP70P72LxvkdRFB2XvcmFEKJak64kWrPZXKHp\niquQUoL0DXV0q5+KYE1wWm1VS+azZ8/m9ddfv/KLAYGIOtGAEZFzDs6fAOMk4EP0+kg6dLjEZ5/9\nQlxcnPorjtpEOgNbHWqKoqjlcbIxQZbu6fX6ckXsQLkKExkdacdWUWRYE5boNalywlXCr0wrt7Ui\nsr4P2nMbjUa/rbYRQryp+f9y9azVmnSBcrqSO2Z/WTEg9yOT5FJVSNK1bkCQ5WUmk4lnZ8/i/y1e\nYvkFvQIN2kJpPuSkgU4PZcWAhXChmGnTJjB37qxymqLWAcyRicKRSNe6Q03KB/J3ZdQit62X16P9\nZzabbZKntr5aErF2zNcKEUvYS9hZk4+fL68dgi0ihoolGrBouDt37uTcuXNucRgzm80kJCQQHR3N\nunXrOHXqFKNHj+bSpUt069aNf/3rX26t7qn2pAvuadvVumgZDAZCQkLUqMNdKCkpoaCg4KoStvT0\ndG6+pS8ZZ9JQAgNAEYi2d8HJn6DwIgQaoCQQRbkTIT5BrzexYcN39OvXTx27ttrBmVpke7CeIIKC\ngtSIBCyEWFxcrPpWaMlD7gYhIZeV1kSsJVDtebVjtCZiWwkb7Tm8ScTuXubbStjZigalTiq9Jrwl\ny3iL6CuakEpKShBCcOTIEVasWMG+ffuIjY0lPj6eGTNm0Lt3b6fPtWTJEjp06KCulp955hmmT5/O\n3XffzSOPPMLy5ct56KGH3HZtfpn2cwbaBglXHghZApOTk4MQFjvH0NBQt9VgSg1Uev5GRESoxwf4\n7vvvaNWqlYVw9TpEmRGR8BAc3wIFFwAFpTQS6IkQ39CjRyK//36YG2+8UT22bDGuW7cuISEhbskQ\ny3ui0+moW7cugYGBqjwgf15cXExwcLAa/dqDJE+DwUBoaCjh4eFERESo+7/JaFneK20kbb0MF0Ko\nx5LdhfJ+yomtoKCA4uJidR+96hwVakk4KCiIkJAQdDrLjtByNSMnwIKCAgoLC9X7aDKZPHLtvlhd\nyHMGBARw77338tprrzFp0iR+/PFHxo0b58puvaSlpbFx40YmTZqkfu/HH3/kb3/7G2Axu1mzZo17\nLuAyakSkC84nhSpLNLmSZLI+vrbiITAwsFyZjclkYu6LL7Bw/oIrv2MyQ/+ZsOP/oDgbhA6dch1m\n0QTYyssvP8/UqU+Sn5+vasJV2UvN+hqt74lWt5U/l9GVwWBwi85dWUQsozrrSM7W8lpRFAwGg/q1\nLa20urX72oO8Hi08XTnh6wSe9vw5OTnUq1eP66+/vlzi2RlMnTqV1157TQ1cLl68SL169dRJPDo6\nmoyMDPcM/jKqPelqa3UdIUlJhrLAOywszCZhVYV05fGlBhoUFKRqomD5w4667x5+2vRfy1Z1egMI\nHQycBckLoTgXaIyiBGM2BwIH2LlzK127dlUTVgBhYWFuSZJB5bptWVkZJSUlBAYGEhYW5rHayIqI\nWEsk0k9Vkogk14CAgKs8OPR6vXosOYFoo2pbSSs/rfssh4rIr7LKicrKt/z92rXvZG5ubpU03Q0b\nNhAVFUV8fDzJycnq8W1N5u5EtSddCUdIUkuGISEhdsu/XCFde4ksebxff/2V2+4cwoWMTHQhQZhD\nW0PuaUi6Fza/AmVBQDRQQmBgIeHheo4fzyIwMJD8/HxVw7PO/LsC2VQij2ut28quo4p0W2/BFpHI\nCFZ2wel0uquiYlvJOqCcXmpNxPaM0qtrVOxo+ZYj1+7rSBeukGB2dnaVSPfnn39m3bp1bNy4kaKi\nIvLy8njyySfJyclRn6O0tDSbbfRVgX9Pa07AHkmaTCby8vLUJFZERESlS2NnSNdsNqvbgAcEBFC3\nbl2bLmbrN6wnISHhCuHG3AV5pyGqHexaAWVtgCDgLHXr6vjqqw9JSzuplsTpdDoiIyOvSiy5AikV\nGI3GCnVbqQ06qtt6C3JyKysro06dOoSHh6sasSy/A9QJw1rb1FaSyOvV6XSqXionZLgyURcWFqpO\nbJLcq3MFgSRXWXESEhJCnTp1bF57QUFBuWv35XVrST8vL6/cbg/O4uWXX+bMmTOcOHGC1atX079/\nf1auXMktt9yiei24aHZjF9U+0rUnL7ia1a/oeNZwtNtLCMHCNxbx0j9etHxDp2Bucz8cXAmYIfNX\nMPYDdgB59O07mH/96z0iIiLIzs4mMDCw3LGrIn1odVsZzRgMhnI1tJKo3KHbuhPajL2tsdlbWltL\nE1p9V0Zy1mRiHRHbKuMCCzl5u83ZExGnrcoJKL9tkCTdgoICnzS0aK+7qpFuRZg/fz6jR49mzpw5\n3HDDDUycONGtx6/2pCuhJSLr8i9XWl/tEZsj3V4SpaWljBz7d777ZgNKgA5BCErHsYiDK8FYePlT\ndwGbgEIWLnydBx6YqGbe3ekAZt00IUvCZMODXLIHBQV5VLd1FlKHLy4udlpTdjcRK4pSzidZ+ldU\n1OZcnf0mJLRar06nU21SvW0Sb/0+5uXluW379b59+9K3b18A4uLi2Llzp1uOawvVnnS1ka5sbJAv\npzv8Ya0jCklciqJU2u2Vk5NDy+tbUZCXj2IIhJBYlNh+FsI1S2P524HvgWJSUlJo2bKl6pxUkeYs\noy5HYe33K3VbRVEICQlRCU37+ZKSknLbwviKNKREALhNU65M49T+A8qRp0zcQXmt2Nqw3pNtzr6U\nNLRdhr4yiZe/W9VEmq9Q7UkXrpQZlZaWEhgY6JboUBvpyISSrbbdinDmzBnatGlz5Xj1umCu1x4O\nrQKzALMZuAXYTKtWzdmy5Uf0ej1BQUFu29VCRmLaiF/eK3l8WU4VGhparrNNvjQy0y1JyltELFcr\nRqOR4OBgt3teWEO7tLblKWtNxPLnAQEB5YhYC1lXa4+Iq9Jd52/Rs3Yyc6XVubLVi3UAJEvGqhuq\nPenKJJNsk3Sn6Y2Mnm217drDunXrGDly5OWDBKJE34Q5uCEc/QqMDYDzWEzkd/D44w8xc+ZMDAYD\nwcHBblnSS/lDuyME4LBua692Vkae8n67m4jl2GV5mrvasF2BNRFryw3lNUsiAcrdA9lcY8tvQhIx\nVL82Z2e1ZHdWTlijNtL1EfR6PaGhoWpSy12QD0Z+fr5TurBKuIGhoKsDjTogTKVw8lswdgb2odfX\n4frrzbz88if07dtX7cpyFPYiXa1uK7VPbRQm620DAgIc1kZt1c5aF+C7g4jlasKX5WkVQSZlzWYz\nderUuWolpb0f8p+8H9ZNHbaIWNsBWZlOWt2hncwkrFcDtjwntJ+Vq0931al7E0oly9RqUQ9TWlpK\nWVkZBQUFVZ75ZCmVTJCEhoaW63Kyh++++57R90ygOPYeOPEphDe11OCWFYC5F5bqhBJmzZrDY489\n6nJzg4wEtVG91G3lxpuBgYHqQ6vVu6WO64mX1x7x2CNiSWiyvtkdJXHugnweZILRmR2SHSVieR4t\nrCM8eSyt94Uka2/q7iUlJWoy0RvQRsRSI966dSsLFizAZDLx6KOP0q1bNzp16kRwcLDDxy0pKaFP\nnz5qm/iIESOYO3euO81uKvxj1BjSNRqNVcpmWrfthoaGUlhYqPa3V4bVq//NI48/TVHnl2D3DBAm\nMJeBsRRF6YYQBwgPD+GLL/5Nz549q1SKJZe4ERER5XTb4OBgDAaDGjHIhFtxcbHPCK0yIpYvU1BQ\nEMHBwX5DtlA+8pZ+B1WFljwrc2CriIhl6ZzBYCh3PHcnrGzB26SrhdzXz2QysXfvXp577jm6dOnC\n3r176d27N2+//bZTxyssLCQ0NBSTyUSvXr1YsmQJixYtYsSIEarZTXx8vKtmNxXe+GovL0D5pJcr\n0HaqaSsSHH1g33t/Kc8+P5+irgshZQqU5UJAKBgNQAxC7GfQoIG8/fZioqOjq/wiaF88R3TboKAg\nQkNDfUJokgCsfXPLyspULVRRFHU5qY2IfVVqpU3iaZsF3AFbCSNbybqKiFj+XH5dUZtzVRJW9iAT\ngL6Colha93v16kVoaCgfffSROi5nIfdWk00fiqLw3//+l1WrVgEWs5t58+a51WEMagjpwhWd0xmh\nvzL/WUeI/NUFrzN/8TIL4f78IBjzQNGDMeLyJ07y7rtvMm7cOHJyctxCIlprP1u6rUx2OaPbegvS\nDctsNpcjNG3kJzd6BK6SJjxJxNokXlBQkNeSeLY084qIWEJWc1jXEctVhCNtzq6YpPuyDVh77uLi\nYkJCQtSfuTIms9lM9+7dOX78OJMnT6ZVq1ZERkZ61OwGahjpOgpHO9Uqa5B47vl/8O5Hayjq/Ar8\nPBGMBZd/GAXkAYUcPvwrcXFx6nGq8tDKSULOymFhYeV8ErS6rb8lorTaqK2KCVsRsTeJWEoJ/nLv\ntEQs752cDORKR0aztnRdLUHbImJtwqq6+E24uxtNp9Oxd+9ecnNzGTZsGIcPH77qM564/hpButoG\nCbksswVH23a1x7VFukIInpz+LP/6KpnCDnMvE24Rls0+I4A/iYqqx4kTmepYrOt+nYG2wy44OJiQ\nkBByc3MpKipSNVp/TkTJ6gZnI29rIraum3UHEWtbi71RD+wstLpyeHi4w9KErSaMyhzY7FUOyP/K\nCd7XcGe5WEREBH379iUlJYXs7GyPmt1ADSFdCW3ZjRbOtO1qYYt0zWYzDz36BF/+sJ/C6DHw0wQw\nhQBmoJjGjYN5443FqglyVaCtpLDWbaVfgiy0l5GRTKL5Q32njLyFEOWaL1yFIw0M9ojYuuxI21rs\ny3pgW5AJ0sp0ZUelCZPJ5BARS98F7c+lNCGJWHYr+sJvQv4Ns7Ozq9QCfOHCBdXTpKioiE2bNvHs\ns8+qZjejRo3yiNkN1BDS1Ua61iTpTNuureNqH0qTycS4CQ+x/r8HKNE3gn0vgCkWOAWUMHToKN5/\n//+p5GjreI4K/tpxW+u2knjkPm6yYsHXXWQSckXhDdMcV4hYp9OpnsTumAzcCXdMBhURsT1PYmu/\nCW25oaxWUBRFTdwKISr0m/BUU4d2lVjVSPfs2bPcd9996qQyatQobr/9dtq3b+9RsxuoIaQroSU1\nacotW1xdWTZqj1dWVsaoseP59vtNCKED4xEwdwL2A2V8+OGH/P3vf3f4eBXBOrkn621t6bbWhGEr\n2jEajeW8FTyZmLImDF8l8SoiYinTlJaWqqsiafZTUUTsTWgbMNw9GchyMkeMf6wbMeRzKycqSX6S\niLXJOk/5TcjzSuTk5FSJdDt37swvv/xy1fc9bXYDNZB0TSZTOVNuZ4rZbR1PLqeGjRjLj/9ZD7og\nMJcC8cB+DAY9+/YdKLcFemXHswVr3bZOnTpX+SRIMnZEe7QV7TiamHKFdDxhTOMuSMKQGrjURm1F\nxDJJ6U0itq6a8FZ5nyNEbO03IaUHbUSsRWBgoEfbnOVnc3JyqF+/flUu32eoEaSrTQLIioTIvcPF\nrgAAIABJREFUyEi31MMWFhZyc98B/HZoP+hDwVQItAQOMXToX1m58uMqRSRa3TYoKMhmva22I6oq\nL6SriSl7iTlvG9M4C3vRoz1pQkZ9JSUl6sTnCSKWKxfwj8nKmoilzCV3KtFGxPKz1n4TtohYG3BU\nRMSVdddp5YW8vDyHAh1/RI0gXaPRqO5cK3c5cAcKCgpo3ry55QtdIJiMQEMgjU8//Yjhw4c7dTzr\nSNeebguer7e1twyXL4ZswJAvhJZwtPum+WMiyl6JWkXQapjaY1nfk6oSsVb39sfJSpvI0+7Gof25\ntTQhn11tgq0iIq7Ib8LWRqTyvmpJt7o6jEENIV25ZJTbnLsDmZmZtGjR4vJXwZetGHVANqdPHycq\nKsrpY0rSdUS3lUt1byd6HFlyyvpQuPLymEwmj7SdugJtmZU7JitH7ok1EQcEBJTTzbWQHZD+2LwC\nV8ZnbzKt6J7YqpyAyh3YwDYRy5I++bwVFhaydOlSLl686PKzlpaWxrhx48jMtJR0PvDAAzz++ONc\nunSJUaNGcfr0aVq0aMFnn33mERezGuG9IDUx6TFQp06dKh3r7NmztGzZ8vJ3GgHZBAToWLLkDSZO\nnODysfPz89UX1ZZPgox+/Hmprh2fjMytIxQt6XizdM2T7buOoCI9VBv5ye/Zih59DVs+HVWFI0Rs\nXcqnhVY/ln/bF198kS1btpCenk7jxo0ZOHAg7733nsNjyszMJDMzk/j4ePLz8+nevTtr167lww8/\npEGDBsyYMYNXX32VS5cuMX/+fFcvvWYb3kjSteW+5QzKyso4efIkt98+lIwMBbM5D/iTtm07sGbN\nag0ROz8+uUzX6/WEhYUB5Wsk5fiddbLyBrRLdXvjs+ch4MnSNetElD/dP3lPpBOeJBBrucaXqwTr\nqhNPGw85GhHLMWgrceTfduTIkaxevZoLFy6Qnp5O//79XR7P0KFDmTJlClOmTGHLli1ERUWRmZlJ\nv379+O2331w9bM03vJH/dUVekMv9kydPcuedf+PcuYaYzceAAh59dDKvvvqyy1GJNBPX6XTlWjgl\n/NknAVBfRkeW6pUV6nuihtjf2netIaNHsPgb6/X6qyJi2YrrCyL2ZJlaRbCVS5BjsSZiqeMKIdi9\nezeNGzfmwIEDHDx4kNDQUNq2bUvbtm1dHsupU6fYt28fPXv2JCsrS5UNmzRpwvnz56t8rbZQI0gX\nXHMa03ownDp1iiFDhvHnnw0R4hBQyubNm2jfvr1LL7L19j4BAQFqC7LcPVZqgL5YClcGqSubzWZV\n6nAF9oi4KjXE/t6+ay+R54hG7Gki1ka3vnSh08K6usZoNKq7DgcEBLBmzRq+//57zp8/T2JiIrNm\nzeL55593OaGWn5/PiBEjWLJkiVM7hVcVNYZ0wbm9w7QeDEePHuWOO4aSlxcJ/EadOiEcP36ayMhI\nLl265NQYtEQeEhJSrt42KChIJV+j0ahGPRUV6PviJXA16+8MqlJDrCiKX7fvgmuJPG8SsfUuGP62\nOtBWdsiAZcOGDfzvf//jww8/pHv37uzdu5c9e/a4XKkkjcvvvfdetdU3KipKjXYzMzNp3LixOy9L\nRY3QdOGKwbG9UhJrD4bQ0FCSk5MZMWI0RUWBQB7jx9/Lu+++o/5OdnY24eHhlT6Y1vW20sXeEd3W\nWt8yGo0IIcplwL1RoK9tIHDXfm1VGY+t+yIhJzB/M/eRZOEp46HKknX2iNiftW8J+Qzq9XrV2GnG\njBnodDoWL17stjKxcePG0bBhQxYtWqR+75lnnqF+/fo888wztYk0RyCjgEuXLlGvXr2rHibZFizN\nVwIDA1m9ejX3338/oAd0bNy49ipBPicnx+aeWBJymSaj1ZCQELu6raNkVlFXkLZhwV1LTVmgL4Rw\nW9banZBkJicsvV7vNOF4enySLLyRiLJ1/sqIWFEUddcHT23XVBXYmrCSk5OZN28es2bNYujQoW67\npz///DN9+vShc+fOqoT18ssvk5SUxMiRI/njjz+IjY3l888/r4qpzrVBumazmT///LMc6UpCkUsV\nWfS+aNFiZs+eCcDTTz/D888/Z1O3zM3NrVBzteXvoBX/tfW2wcHBVe5c03ZKuYNwvGlM4wocib6r\nEvm5A9qluj9NWPK+SN1cTtq+nqBswXpbpKKiIubMmcPFixd55513aNSokU/H5yJqdvUCXF3BIAlF\n62UgifCBBx7h008/Rq8PYNOm/3DjjTfaPa71xGSt28pec9m3D6jdPO5K8jjSKaU1tdZKE9b6sDaJ\n4q9VE45m1Z1p5nBnDbGv/BIchRxLWVkZUH5naDlxW98XbxOxNn8gg5KdO3cyc+ZMnnjiCcaMGeNX\n99RdqDGkq4UkW2uj8vz8fDp27ExW1lluvXUwq1b9q0IbRgkt6Von4CrzSfB0kqciwrFXoiUTUeB/\ntobgnkSeK/fFmQSmv/klWMOazLSTvr374k0ilmWaMtlYWlrK3LlzOXLkCGvWrKFZs2ZuO5e/ocbI\nC3KJkp+fj16vv0qHPX36tFrPt2zZcu65Z6xDx5UlK3q93u26rbcgI1ttO6WnDFyqAk/svmsPzjZz\n+LtfApQnM1fvYUX3RUvEsmXX2eu3NSHs37+f6dOnM378eCZNmuTz59BNqPmabn5+PgUFBSiKorbY\nSnz++efce++9ABw5coTY2FinjitlA6nt2tJt/TkJJZfB2iSPtQ4qr9G6YsIbpOLr9l0ttDXE2gJ9\nuTTX6/UEBwf7hRaqhb3o1l3Hr2q3oXaFILc+X7hwISkpKbz33nsud3z6KWo+6co6xoKCAgIDA1Vf\ng8GDb2Pr1i3cdFMv/vOfHxxeCkpNsaSkRNU9q5NPApTv1qosY+2LhFR1KGGSz4HRaFR3TKiohthX\nEZq3VwgSjhKxlLRKSkpUyei3335j6tSpDBs2jMcff9zvJBo34NpIpFl3pS1btoytW7ewYsUKRo0a\n5dBxtLqtwWAgODi4XCQI3tVtXYErHrf2ElLWiRd3NHL4e/uutR9BREREubpqLeG42xDemTFqy6y8\nvUJwpu0bYP369RQXF3Pq1ClSU1N5//33ad++vVfH7A+oMZGu3DxP+tOGhISoxfSOLPkrqreVL542\nwtXpdBgMBr+Lbr0ROVbUyKGtCpDRja1z+3v7LpRfBjta0+rtlYK2icDfcghwdZuxXq9nxYoVfPHF\nF/z+++/k5+fTuXNn3n33XW644QZfD9cTqPnygrbbTDZAOApt44TUZbUb85nNZvXn2iWmNurTNiv4\ngkS0xjRSc/QWKmvkkP9kstEXDQSOwN0t0J4gYl9Ht45ASjLyfVIUhQ8++IDVq1fz9ttvc8MNN5CX\nl8fevXvp1KmTx7fdadGihVrFFBgYSGpqqkfPdxk1X16QsK4qsAdJpvIBlvW20ogG7NfbapsVbC0x\n3dk1Zu8aZD1rVYxpqgJ72wBJovVn03Nwv/E5uL+G2BFzcV/COro1GAxkZGTw+OOPEx8fz3//+181\nwR0eHk6fPn28Mi6dTkdycrLf7DRRY0jXujnCHqx127p165ZLkMAV3bYy9/zAwMAK99eShtDuKLWx\ndQ3u2jvN3dDq61J+MBgM6grClnmLL0zPwTt+CVq4UkOs01m2jDcajXZb0n0JaxMdRVFYtWoVS5cu\n5Y033uDGG2/02fMpJzp/gf/99aoIe5GuteGNJFMt2Uo9T9b6OrNMt9c1Zi8Zpd1htTJYt8b6YzeZ\no2P0tNeuo2P0deRoLyEld0SRKC4u9vi9cRYyApeT//nz55k2bRrR0dH897//ddueha5CURQGDx6M\noig8+OCDPPDAAz4dT40iXUW5sv+SNawNb2TUJRNkWh3KnZ1a9iIbe36ytiIurTGNP3aTQXkf3srG\naI9sKvParcpE4wvjbmchVzIycgwICLB5b2y50Xkrr6Dd3ic0NBS9Xs+6detYtGgR8+fPp3///j6f\nEAC2b9+umpIPHDiQ9u3b07t3b5+Nx/+etirCWl7Q6rbSlKYi3dZbpi+2yEa715itHXjl8jI4ONjv\njGnAfUkoe/fGlnZubwNIW2P0Z78EsG8u7uy98WTpmlZfDgsLIzs7m6effprg4GA2bdrkkQ0dXUWT\nJk0AaNSoEcOGDSM1NbWWdN0FbZ2uI7qttlPL14kJW8koW8tL6abmb8tLR7f0cQW27o2z5t7+7pcA\nrpmL20tieoKIhSi/Nbter2fz5s28+OKLPP/88wwZMsTnz6MW0gUwLCyMgoICfvjhB+bOnevTMdUo\n0pUQQpCTk6NuzW5Lt5Uk4c8voCRb6+WlJBpPLL1dGaO323crqwowGo3lqgLkWGUE7o8auLu2zpGB\nhy0ilnkFuV28s6VrUqKT71V+fj6zZ8+moKCAb7/9loYNG7o0Zk8iKyuLYcOGqUndsWPHMmjQIJ+O\nqcbU6YJFJsjLy8NkMhEWFmaz3lZqor4qr6oMzjYPaCM++VJ52iGqOrTvwpUlsCQhqeH7S201XJG/\nwPFGDHfAmRpi4KoKj59//pnnnnuOadOmMWrUKL/8+/sY10adrtQ8pfGNVrf1Z7NuuNqYxlG5o7Kl\ntztbd8H/23fBfgTuKw3UGtq/ty+eSWdqiOXnv/zyS9q1a8eaNWtIS0tj7dq1XHfddV4bc01BjYp0\nS0tL1R1EjUajSiwmk0k1wfFHknDGmMYVVNS668webK74OXgb1n4JjnS9WTdyeMPox5U2Y29D5kRk\nYrS0tJRHH32UXbt2ce7cOeLj40lMTOTNN9/0O7nGT3BtRLoPP/wwZ8+epVu3boSFhfG///2PV155\nhdDQULXMxpubPVYGbxGZI1nviqwdgXJE5uuEY0XQltM5E4FXpIHaS9S52sjh7jZjT0HbnRceHo7R\naGTJkiUUFBSwbds2GjZsyN69e/ntt9+8mj9ISEggOjqadevWeeWcnkKNinSFEGzfvp3HHnuMtLQ0\n+vTpQ3p6Oq1btyYxMZGePXvSqlUrAJv6p7u6xRwZp79pohVpfIDa8BEYGOgX1RJaeIvIquonazJV\n3Vzc07DlyXvo0CGmTp3KqFGjmDx5ss/G/cYbb7Bnzx5yc3OrC+leG5Guoijk5+dz//3388gjj6iG\n47///js7duzg/fff59ChQxgMBrp160ZiYiJJSUlERkba1D+1EY27oC2v8idNVKvxyaWlyWRSSUwm\nfOT9sd6DzRfwhF9CRajMxrCiahKdTkdZWZnHzMXdBe2kEBYWhtlsZvHixWzatInly5eru674Amlp\naWzcuJHZs2eX2zK9uqJGRbqOQAhBfn4+u3fvZseOHezcuZOsrCxiY2NJSEigR48edOzYUTVlcbUQ\n3xqycsJkMnmlx98VWLfv2rIMtOcoVpX74+w4/XXbHFvVJIB6b7xhguQMbEW3x44d48knn2Tw4ME8\n9dRTPu/Yu/vuu5k9ezY5OTm8/vrrtZFudYOiKISHh3PLLbdwyy23AJYX5fTp0+zYsYMvv/yS559/\nHiEEXbp0ISEhgZ49exIVFaV69kpZwt6OuxL+bEyjhaPtu842KrjbyEaWgWlrsP0J8jqNRqPq/BYQ\nEKDeH1smSN6YqGxBm9ALCwsDYOnSpXzxxRe88847dOnSxavjsYUNGzYQFRVFfHw8ycnJlZpZVQdc\nc5GuI5BEuXfvXlJSUkhJSeH06dM0bNiQxMREevToQXx8PEFBQaqZDVxddqQ1z6kOOp67NNHK9E9X\n6mO13Vr+uBedhCPm4tYTlSRod5X1VQZb5WppaWk89thjJCUl8fzzz5czbfIlZs2axcqVKwkICFDr\n8IcPH87HH3/s66FVhppvYu5pCCHIyspSSXj37t0UFRXRrl07VZaIi4tDCEFubq5KXlIH9FaSzhlo\n9WVPTwrajihrWcLestsfk462UFVzcU9MVLZg3YyhKAqffPIJH330EYsXL6ZHjx4uH9vT2LJlS628\ncC1BURSaNGnC0KFDGTp0KGCJag4ePMiOHTt48803OXz4MNnZ2Zw/f55XX32VwYMHq3useSNJ5yh8\n0b7rivewoiiUlJT4dSMGuEfyqChRZ88k35lEpraGWUa3WVlZTJ06lZYtW/Ljjz8SEhLi9Lhr4Txq\nI1034eTJk9x888306dOHoUOHcvjwYXbu3Mmff/5JXFycWrLWtm1btWHDOsniaW3P36NGrfewthtK\ne2+c8R72NLTmL96YvKzNbCQh26qv1t4f6+1zdDoda9as4c0332TBggX07dvXL+5nDUOtvOBpmM1m\n9u/ff9Ume2azmePHj7Njxw5SUlL43//+h16vp2vXrqo+3LBhw3Ianye0PU93vbkL1pqonKC00gRU\n7j3saWitDX2531tlHgpyrIGBgYSEhHDp0iWmT59O3bp1WbhwIRERET4Z9zWAWtL1FwghKCwsZM+e\nPaSkpJCamkp6ejpNmjRR64a7dOlSLuMNrpNMdWjfBcclD1+07doap7b0z98gZQlZjaLT6Rg4cCB6\nvZ7MzEzGjh3LpEmTaNOmjV8md2sIaknXnyGEIC0tTU3S/fLLL5SWltKpUye1ZC06OrpcRFNZSZYr\nPgS+gDvGaW3r6Gy3mLfG6S1ot88xGAzk5eUxc+ZMysrKaN26NQcPHiQ1NZWNGzfSoUMHj46lpKSE\nPn36qK3mI0aM8Jqf7dGjR0lNTWXs2LFeOZ8Vakm3uqG0tJQDBw6oRHz8+HEiIyPp3r07PXr0oHv3\n7oSEhFyV6ZZRcGlpqd9LCdraYHdHjba0T3AtCVVdytWso3C9Xs+2bduYM2cOM2bMYMSIET6ZKAoL\nCwkNDcVkMtGrVy/efPNNkpKSPHpOs9nM559/zi+//ML8+fN9cd211QvVDUFBQSQkJJCQkMCUKVMQ\nQnDx4kV27tzJjh07eOutt8jNzVV9JXr06EHTpk1JSUmhe/fualRXUlLi0wJ8W/CGraG2GkBu+61d\nKThi5O1Oc3FPw3r7nKKiIubNm0dGRgbr168nKirKZ2OTG1OWlJSoxkruxMaNG2nRogUdOnQgLS2N\n8PBwdaeYo0eP+t3frDbSrcaQvhLbt29n9erV7Nixg4SEBJKSktT/RkZGqhUBVdmF2F3Q+iX4umHE\nXpOCbAP3501A4ertcwICAkhNTeWZZ55h8uTJ3HPPPT7Xbc1mM927d+f48eNMnjyZV155xW3HPn/+\nPA899BD9+/dn4MCBfPDBB+zfv5+vv/6a4OBgbr31VubOnUufPn3cdk4HUbMi3e+++44nn3wSs9nM\nxIkTeeaZZ3w9JJ9Ar9fToUMHTpw4QWZmJhs3bqRbt26qr8Snn35KVlYWMTExapKuU6dOKIpi06DF\nk74A/uiXYMvIW+7cUVpaqpJVQUGBX1mCSlhvn1NaWso///lPfv31Vz7//HNiY2N9PUTA0hq9d+9e\ncnNzGTp0KIcOHaqylix3g2nUqBEjRoxg27ZttG7dmldffZXJkyczc+ZM6tevz1/+8hf1byV/x9eo\ndpGu2WymTZs2bN68maZNm5KYmMjq1atp166dr4fmM8hozVbGX+srkZKSwv79+1Vfie7du9OzZ0+a\nNGniVJLOWWibBypqjfUHVGQubl2SVZH3sLdeaFvdbwcOHGDatGmMHTuWRx55xG/v8T/+8Q/CwsKY\nNm2aW463aNEi1q9fz6VLl6hfvz6ffPIJDRo04NSpU0yePJmUlBS3R9cOouZEuqmpqbRu3ZrmzZsD\nMHr0aNauXXtNk640oanoZ3FxccTFxTFmzJirfCXmzZtXzlciKSmJG264Ab1er24H76qdozYB5e9L\ndHsaszMmP46YIFUF2jpmaTD+2muvsXXrVlasWEHr1q3der6q4sKFCwQGBlK3bl2KiorYtGkTzz77\nrFuOvW3bNj7//HN+/PFHsrOzeeyxx/j3v//N2LFjad26NUuXLuXLL79k+/btXLx4kQYNGrjlvFWF\nf74FdpCenk5MTIz6dXR0NKmpqT4cUfWCoigEBwdz4403cuONNwLlfSVkf3thYSHt2rVTk3TSV8Le\nLhPaBJS2882fE1DWTluOTCYV7S+mjYSrMlnZgq3o9vfff+fJJ59kyJAh/PDDD35ZpXL27Fnuu+8+\ndVPQUaNGcfvtt1fpmD/++CP9+/fHYDAQERFBaWkp1113HQ8//DBz586lTZs2DBw4kObNm/PXv/6V\nQ4cOUbduXTddUdVR7UjXlhziry90dYEjvhJHjhyhTp06dO/encTERBITEzEYDJjN5nK+EjqdTiVl\nf49urX1kq/IcObIlkqvew9Zm7UII3nnnHdauXcu7775Lp06dXB63p9G5c2d++eUXl3/fWoctLS1l\nwIABrF27lh49ehAbG8uvv/5Kr169GDBgALNnz2bFihW0atWKNm3asG3bNpKTk7lw4QJNmjRxxyVV\nGf75RthBdHQ0Z86cUb9OS0ujadOmPhxRzURAQABdu3ala9euPPzwwwghyMnJITU1lR07drB8+fJy\nvhLx8fHs37+fhIQE2rRpo3beeSNJ5yysd0nwlP5ZVe9hWxPD6dOnefzxx+nduzc//vijV8yKfAFJ\ntvJ5OX78OOHh4TRu3JjVq1czZcoUTpw4Qbt27fjoo484ePAgBoOBevXqMWHCBNq0aQNA8+bN2bhx\no98QLlTDRJrJZKJt27Zs3ryZ6667jqSkJFatWkX79u19PbRrDtJX4oMPPuDdd98lLi6OqKgo2rZt\nq8oSjRo1sukJ4G5zc0fg7ujWXWOyZemo0+nUUr+8vDyio6NZuXIlK1euZMmSJSQmJvp03J7E+fPn\nOXHihGozmZyczFtvvcX48eO57bbb0Ov13H333RgMBlauXMm6dev47LPPKCwsZOHChbRs2dLHVwDU\npESaXq/nrbfeYtCgQWrJmLcINy0tjXHjxpGZmYler+eBBx7g8ccf98q5/RE6nY7Y2Fi2bNnCxx9/\nzF//+tdyvhLPPvssGRkZNGnSRK0b7tq161VJOm+UY3lzPzVnYC1LaLc+DwgI4MSJE9x1112YTCYa\nNmzIvffeS35+vo9H7RnI7awKCgr44Ycf+OSTT9i7dy8ffvihunNETEwMXbp04b333qNJkybcfvvt\njBkzhgEDBqhNGHLS8ldUu0jXl8jMzCQzM5P4+Hjy8/Pp3r37NV85AfbrHyvzlUhKSqJ58+ZqVOdu\n85qqmot7E9qknkw+fvHFF7z99ts8/fTTmEwmdu3axYULF1ixYoXHx+PNIOOzzz7j3Xff5YknnmDo\n0KH85S9/Yc+ePcydO5fJkydz8eJFnnnmGbp27cqECRMoLS1l1KhR6PV6Nm7ciBBCXR34CeHWei94\nAkOHDuWxxx7j1ltv9fVQqhVKS0vZv38/O3fuVH0l6tatW66bzpavhLOddNrmAX+uD7ZVsnbx4kWm\nTZtG48aNefXVVwkPD/f6uLwRZJjNZiZPnszBgwd5+eWX1QTtpk2b2LRpE02bNuVvf/sbzZo1Izk5\nmVWrVpGZmcnRo0eZP38+d955p9vG4mbUkq67cerUKfr168evv/6qbupXC9dg7Suxa9cu1VdCboUk\nk3OO7EDsbXPxqsB6+xydTseGDRt47bXXeOmllxg4cKDPdWcJTwQZly5dYtq0aSxdupSAgAB14gHY\nu3cv77//Pj179uS+++4D4MyZM+zcuZO4uDgSEhIAy+Tqh1UytaTrTuTn59OvXz/mzJnDXXfd5evh\n1EhIXwnZSXfo0CEMBgPdunVTmzjq1at3VZJOlqxJ+0V/jm6tt8/Jzc1VW9qXLFlCvXr1fDzKK/BU\nkJGWlkanTp1ITU3l4MGD7N69m1OnTlFcXMzy5ctZs2YN+/btIyAggMzMTBYsWECzZs0Ay4TlL7uI\n2EAt6boLRqORIUOG8Je//IUnnnjC18O5ZiCEIC8vj927d5OSksLOnTvJzMwkNjaWxMRE2rdvz9at\nWxk1ahSxsbGYzWaEEH7pmWBr+5zk5GTmzZvHzJkzGTZsmF8RiaeCDKm/vvLKK3zzzTekp6dz3333\nUa9ePbZu3YqiKHz11Vd8/fXXfPzxx0yePLk6SXm1pOsujBs3joYNG7Jo0SJfD+Wah/SVWLx4McuX\nL6d79+7o9Xrat2+vyhJNmzatMEnn7R2ara0iDQYDhYWFzJkzh4sXL/LOO+/QqFEjr4zFUbgzyJBb\nUUlok15//PEHjRs3Vj2Ly8rK6NOnD99++y2RkZHq7/iLaY0DqCVdd+Dnn3+mT58+dO7cWV3WvPzy\ny9x2221eOb/ZbCYhIYHo6Ojqsg21x5GTk8OQIUNYsGABPXv2LOcrkZKSUs5XIjExkW7dumEwGOxu\nc+6JaNjah0Kn05GSksLMmTN54oknGDNmjF+SibuCDC3B7t+/nw4dOtjV2qdMmUJ+fj7Lli1T9Vo/\nqkxwBLWkWxPwxhtvsGfPHnJzc2tJ10EIIcjMzFQlid27d5fzlUhKSqJly5YOJ+lcgfX2OSUlJbz0\n0kscOXKE//u//1M1Sn+DO4IMbWR6+PBhJk6cSN++fXn66aepX79+uc+eO3eOTz75hE8//ZSEhAQW\nL16sJtWqIWpJt7ojLS2N8ePHM3v2bBYtWlRLulWA1lciJSWFI0eOEBoaSvfu3UlKSiIxMZGIiIir\nknTOOojZ2sRy3759TJ8+nfHjxzNp0qTqFLk5DW1kmpOTQ//+/XnuuecYNmwYcEVu0H7u7bffJiYm\nRi0Fs5YkqhFqSbe64+6772b27Nnk5OTw+uuv15KuG2HtK7Fz585yvhJJSUm0b99e3U3CaDQC9vdb\ns96i3Wg0snDhQlJSUvi///s/WrVq5avL9Sr++OMPHnroIRYvXsySJUto1qwZUVFR7N+/H4PBwBNP\nPEF0dLRNcq1mcoI1ak4b8LWIDRs2EBUVpbZCVjJR1sJJKIpCZGQkgwYNYtCgQYDlhT927Ji6A8eB\nAwfQ6/XEx8eX85WQO01ok3RSKw4KCiIkJITDhw/z5JNPMnz4cL777rvqGrk5BC1Rbt7JRA2cAAAM\n7UlEQVS8mUWLFnHHHXfQsmVLBgwYwHvvvceIESNo164dR48eJTk5mXvuucfmPanGhGsXtZFuNcCs\nWbNYuXIlAQEBFBUVkZeXx/Dhw/n44499PbRrBtI1TfpK7Ny5k/T0dJo0aaIm6UwmE1lZWdx2221k\nZ2eTkJBA69atuXDhAk8//TQjRoy4JhzxioqKCAkJYcGCBbzwwgucPHmSxo0bl/vMyZMnmThxIvPm\nzfPF/mXeQK28UFMgTcZr5QXfQ/pKJCcns2jRIo4fP06fPn1o1qwZzZs3Z9OmTXTo0IFGjRqxa9cu\n9uzZw4kTJwgJCfH42CZOnKjuAnzgwAGPnksmy4QQXLp0ifbt25OSkkLz5s35+9//TlxcHPPnzwfg\n4sWLfPbZZ7zxxhtMmzaNhx9+2KNj8yEqJN2aGb/Xwu3Iycnh7rvvpn379nTs2JGdO3f6ekg+h6Io\nxMTEcOzYMTp37szp06f56quvmDRpEpmZmUydOpW33nqLuXPnsn79ejIyMrxCuADjx4/n+++/98q5\nFEXBaDRSWlpK/fr1GT16NPfeey86nY6nnnqKQ4cOsWHDBgCCg4Np164dmzdvVgn3mpPLhBD2/tWi\nCli+fLnIysry9TDcgvvuu0988MEHQgghysrKRE5Ojo9H5D8wGo2+HoJNnDp1SnTu3NkjxzaZTEII\nIcxmszh+/LiYOnWq2LJli/rzjh07ihdffFEIIcSyZctE3759xblz58odw1/vm5tQIa/WJtI8iEmT\nJnH8+HH16wMHDpCbm0vPnj390aCjQuTl5bFt2zY++ugjwLKrREREhG8H5UeoyYkxa4jLUoKs5NDr\n9TRv3pzi4mK2bdtGTEwMcXFxrF69moSEBEaOHMk999xDdnb2Vce6lu6bFrXygodw5swZmjZtSlxc\nHGCpDd22bRvvv/++mpWVW7cIP19enThxgoYNGzJ+/Hi6devGgw8+qPq+1uLagqxN/vbbbxkwYACz\nZs1i3bp1/POf/+TgwYPs2rULs9lMp06duP766xk7diwGg4Hp06f7XYuzr1BLuh5CSkoKZrOZc+fO\nAagesG3btlVJV5YYaYvszWazT8ZrD0ajkV9++YXJkyfzyy+/EBoaqiZGalHzcfToUV566SW1Pnnj\nxo3MnDmT6dOnk5SUxJQpUzh58iTjx49n/fr1LFiwgIkTJzJhwgR1dVSLK6g+a9xqhu+//57w8HAm\nTJhAZmamuiR/9NFHAfj6669ZtWoV8fHxjB49Wo2IrWsT/aEjJzo6mpiYGNW/dMSIEbz66qs+HVMt\nKofUEKuKEydOcOzYMT799FPGjRvHn3/+yZgxYxgyZAgAhYWFPPjgg+zZs4fAwEA+++wzGjZsyLRp\n04Bq3+TgdtTeCQ9h69atfPXVV6xfv55du3Yxfvx4QkJCuPnmm3nttddYu3Ytd9xxB8eOHWPZsmUA\n/Oc//+Hjjz8mJydHPY4twpXRsLdkiaioKGJiYjhy5AhgKXrv0KGDV85dC9cwZswYbrrpJo4cOUJs\nbCwffvihU79/4cIFwPKM3XzzzQwfPpytW7eSlpaGEII1a9aonx05ciQtWrTg9OnT9OvXj7feekud\nlGsJ1wbsZdm8m+yrWahfv74oKChQv/7222/FpEmThBBCtGvXTvz0009CCCGKiorETTfdJE6dOiVe\nf/118dRTT4n09HQhhBB33323+OSTT4QQQhw9elTk5eXZPJfZbPbkpQghhNi3b59ISEgQXbt2FcOG\nDRPZ2dkeP6cWixYtEh07dhSdO3cWY8aMESUlJV49/7WE/fv3i+DgYJGfn69+b/HixaJNmzZizpw5\nQgghEhMTxZw5c4TRaBRr164VAwYMuOr59MZz6ceokFdrmyM8gG3btjF48GB1G5bS0lKWLl3K2bNn\nmTt3LtHR0WRlZVFaWkpQUBANGjQgPT2dmTNn0qlTJ8aOHUtwcDDNmjVjzZo1tGzZkoEDBxIVFcXx\n48f55ptvUBSF4uJiunbtWuE4tm/fTn5+PklJSeU8SasbMjIy6N27N7/99htBQUGMGjWKO+64g3Hj\nxvl6aDUWo0ePJigoiI8//piJEydy6tQpRo4cyaFDh7jllltISkri0UcfJSAggDNnzvDKK69UJ4Nx\nb6DWe8GbaNmyJatWrVK//vPPPzl69CgxMTEUFBSQkJDAyZMniYuL47vvvqNZs2YEBwdz4cIFYmJi\nCA4OpqSkRCXMlJQU0tPTWbVqFSEhIWRkZPDWW2+RmZnJmTNnmDFjBvfffz+BgYGqBjxv3jzOnDlD\nRkYGf/zxB127dmXhwoVERESQl5fHdddd58M75Dzk1tw6nY7CwsJrop3Wl1i9ejWxsbE0bNiQGTNm\nsHz5cgBWrlzJhg0baN26NV9//TUnT54kJiamWpVA+hq1d8oDaNasGc2aNVNrGhs1asSECRPQ6/VE\nRkZyxx13MHLkSDp06IBOp2PGjBlkZ2ej0+nUXV83bdqk9qv/9ttvDBw4UN2Fdc+ePVy4cIHNmzdT\nVlbGqVOnVENoqQEvXLiQo0ePquSanJxMw4YN2bp1K0uXLuUf//gHbdu29fd9pgBo2rQp06dPJzY2\nltDQUAYNGsSAAQN8Pawaj08++YTbbruNGTNmqN/r27cvhw4dYvXq1cydO5cWLVqgKIpfJHyrC2oV\nbg9AJrokken1erp06ULHjh0RQvDoo4+ydOlSevXqxZQpU7jnnnuIjIykQYMGLF68mNTUVF544QX6\n9+8PwLFjx2jfvj1g0eC7du1KnTp1mDdvHunp6bRu3brc+QsLC+nduzfvvfcev//+OwA333wzQUFB\nXLhwgUaNGtGiRQsAr25X4yqys7NZu3Ytp0+fJiMjg/z8fD799FNfD8tj+O6772jXrh1t2rTxaZXI\nzTffzOjRoxk8eLD6vZiYGMaPH8+cOXPUMki4dhsdXEEt6XoAtrK1UjuXD2l8fDwPPvgg3bt3Vz8z\nYcIEmjdvzrZt28jKyuLWW2/FaDSq25uApTusVatWfPXVV5SVlTF16lSOHj1a7lyhoaHMmjWLn376\nieeff55vvvkGvV5Pbm4u6enp1K9fH4PBQG5uLm+//TabNm0iLy/P5rXIDR59iU2bNtGyZUvq16+P\nXq9n+PDhbN++3adj8hTMZjNTpkzh+++/5+DBg6xatYrffvvNZ+NZvnw5x44d4/XXX1e/17p1a4KC\ngvyyprw6oJZ0vQTraFIIcdVD26lTJxYsWMD06dM5ffo0I0aMoKioSE1cAJw+fZoXXniBzZs3M2LE\nCHQ6HRkZGeoxJfr06cPKlSuJj4/nqaee4ocffsBoNJKRkUHXrl05evQob7zxBmVlZaxYsYL777+f\nixcvAhb99MSJE4DtSFh20nkLsbGxpKSkUFxcjBCCzZs3q5F/TUNqaiqtW7emefPmBAYGMnr0aNau\nXevTMS1btoxdu3ZdNfnWloK5hsqqF2rhRSiKosOS9TSLCv4wiqLUAx4AbgRigFeA9UKIEkVRFCGE\nUBQlXgixT/M7W4H3gFRgFvBP4G/Aw5f/fw0wDzgGLANGAMOAdsAmYDqWSpYAIUSxu6/bESiKMhcY\nDZQBe4FJQogyD55vOTAEyBJCdLn8vXrAv4HmwClgpBAip8KDuHbevwGDhRAPXv76HiBJCPG4O89T\nC9+hlnT9HIolzFSEEA6v5RRF+QTIAX4E0oCNQAIQB4wBJmEh4HVACJAE9AKGA9HA7cBcIcQBRVHe\nBL4BDgNzgEgsBLwB+EwIUeKGy/Q7KIrSG8gHPtaQ7qvARSHEAkVRngHqCSGedfN5RwCDrEg3UQhR\ntf3P3QBFUfRCCO8tcWooaqsX/ByXI95yM6OiKPrLP6voBXgZ+AsWEo0ERgshTiiKcidQdjkaPg9s\nFkJsv3xMnRDCrChKMnAJeF9RlFKgN/ATEA7cCrwL7AOewxLtbXPj5foNhBA/KYrS3OrbdwF9L///\nCiAZcCvpYpkkYzVfRwMZbj6HS6glXPeglnSrISp7+IUQB4GD2u8pihII/IqFUAFeB55TFOXTy98v\nVBTlFJAHzBJCHFYUpRHQFovs0AfYASwTQuQoijIBi/xQI0m3AjQWQmQBCCEyL98fd2MXcP1lwj+L\nRVL5uwfOUwsfoZZ0ayAuSxI6uELQl/XPTZqP/Qw0wqJbTgRWCSHeVxRlHfCUoigPCiHOA+cva80x\nQN5lwg0F9mCJfmvhRgghTIqiTAF+wPI3XC6EOOzjYdXCjagl3RqIy5LEVdGwlBAuf6YYWH35nypZ\nAN8CicA+RVHMwELgM6AL0PDyZ5oCPYCVHrwMf0SWoihRQogsRVGaAOc8cRIhxHdYVhi1qIGoJd1r\nCNbJOFktIYQwXY6wFCFEGiCTOC2BP7Fk67OBKEVRNmKJwH7BkkyryVAo30O/DrgfeBW4D/BtLVct\nqiX+P4HoBi4SqEQXAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fea5d3c4b50>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#3D plot:\n",
    "\n",
    "##Uncomment to plot out-of-notebook (you'll be able to rotate)\n",
    "#%matplotlib qt\n",
    "\n",
    "from mpl_toolkits.mplot3d import Axes3D\n",
    "fig = plt.figure()\n",
    "ax = fig.gca(projection='3d')\n",
    "\n",
    "surf = ax.plot_surface(xx, yy, 100*allOutputs.reshape(100, 100), \\\n",
    "                       cmap=plt.cm.jet)\n",
    "\n",
    "ax.set_xlabel('Hours Sleep')\n",
    "ax.set_ylabel('Hours Study')\n",
    "ax.set_zlabel('Test Score')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def computeNumericalGradient(N, X, y):\n",
    "        paramsInitial = N.getParams()\n",
    "        numgrad = np.zeros(paramsInitial.shape)\n",
    "        perturb = np.zeros(paramsInitial.shape)\n",
    "        e = 1e-4\n",
    "\n",
    "        for p in range(len(paramsInitial)):\n",
    "            #Set perturbation vector\n",
    "            perturb[p] = e\n",
    "            N.setParams(paramsInitial + perturb)\n",
    "            loss2 = N.costFunction(X, y)\n",
    "            \n",
    "            N.setParams(paramsInitial - perturb)\n",
    "            loss1 = N.costFunction(X, y)\n",
    "\n",
    "            #Compute Numerical Gradient\n",
    "            numgrad[p] = (loss2 - loss1) / (2*e)\n",
    "\n",
    "            #Return the value we changed to zero:\n",
    "            perturb[p] = 0\n",
    "            \n",
    "        #Return Params to original value:\n",
    "        N.setParams(paramsInitial)\n",
    "\n",
    "        return numgrad"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}