Assignment2_part1.ipynb

{
 "cells": [
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "# **Part 1: Manual training of multi-layer feedforward network (Example-by-example training)**\n",
    "\n",
    "In this part, I manually trained a multi-layer feedforward network using **example-by-example training**. The training process consists of three main steps:\n",
    "\n",
    "1. **Training the network with the first input $x_1 = 0.7853$ and target $t_1 = 0.707$**.\n",
    "2. **Training the network with the second input $x_2 = 1.57$ and target $t_2 = 1.0$**, using the updated weights from the first input.\n",
    "3. **Calculating the Mean Square Error (MSE)** for both inputs after training."
   ],
   "id": "f70d81c4286928"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "## **Initial Setup**\n",
    "\n",
    "- Initial weights after epoch 1:\n",
    "  - $a_0 = 0.301444$\n",
    "  - $a_1 = 0.201954$\n",
    "  - $b_0 = -0.0844103$\n",
    "  - $b_1 = 0.409993$\n",
    "- Learning rate: $\\beta = 0.1$\n",
    "\n",
    "The network will be trained using two inputs and their corresponding target values:\n",
    "\n",
    "- Input 1: $x_1 = 0.7853$, Target: $t_1 = 0.707$\n",
    "- Input 2: $x_2 = 1.57$, Target: $t_2 = 1.0$\n"
   ],
   "id": "40ff629e7db2a714"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-09-29T22:26:53.334425Z",
     "start_time": "2024-09-29T22:26:53.324958Z"
    }
   },
   "cell_type": "code",
   "source": [
    "import numpy as np\n",
    "\n",
    "# Sigmoid activation function\n",
    "def sigmoid(x):\n",
    "    return 1 / (1 + np.exp(-x))\n",
    "\n",
    "# Derivative of the sigmoid function\n",
    "def sigmoid_derivative(x):\n",
    "    return sigmoid(x) * (1 - sigmoid(x))\n",
    "\n",
    "# Given initial weights after epoch 1\n",
    "a_0 = 0.301444\n",
    "a_1 = 0.201954\n",
    "b_0 = -0.0844103\n",
    "b_1 = 0.409993\n",
    "\n",
    "# Inputs and targets\n",
    "x1 = 0.7853\n",
    "t1 = 0.707\n",
    "x2 = 1.57\n",
    "t2 = 1.0\n",
    "\n",
    "# Learning rate\n",
    "beta = 0.1\n",
    "\n",
    "# Print the initial conditions and setup\n",
    "print(\"=== Initial Setup ===\")\n",
    "print(f\"Initial weights: a_0 = {a_0}, a_1 = {a_1}, b_0 = {b_0}, b_1 = {b_1}\")\n",
    "print(f\"Input 1: x1 = {x1}, Target 1: t1 = {t1}\")\n",
    "print(f\"Input 2: x2 = {x2}, Target 2: t2 = {t2}\")\n",
    "print(f\"Learning rate: beta = {beta}\\n\")\n"
   ],
   "id": "7cd82b304ac4527f",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "=== Initial Setup ===\n",
      "Initial weights: a_0 = 0.301444, a_1 = 0.201954, b_0 = -0.0844103, b_1 = 0.409993\n",
      "Input 1: x1 = 0.7853, Target 1: t1 = 0.707\n",
      "Input 2: x2 = 1.57, Target 2: t2 = 1.0\n",
      "Learning rate: beta = 0.1\n",
      "\n"
     ]
    }
   ],
   "execution_count": 3
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "## **Step 1**\n",
    "\n",
    "### **Part 1(i): Weight Adjustment for the First Input $x_1$**\n",
    "\n",
    "#### **Forward Pass**\n",
    "\n",
    "1. Compute the weighted sum for the hidden layer:\n",
    "   $$\n",
    "   u_1 = a_0 + a_1 \\cdot x_1 = 0.301444 + 0.201954 \\cdot 0.7853 = 0.460038\n",
    "   $$\n",
    "2. Apply the sigmoid activation to get the hidden layer output:\n",
    "   $$\n",
    "   y_1 = \\frac{1}{1 + e^{-u_1}} = \\frac{1}{1 + e^{-0.460038}} = 0.613023\n",
    "   $$\n",
    "3. Compute the weighted sum for the output layer:\n",
    "   $$\n",
    "   v_1 = b_0 + b_1 \\cdot y_1 = -0.0844103 + 0.409993 \\cdot 0.613023 = 0.166925\n",
    "   $$\n",
    "4. Apply the sigmoid activation to get the network output:\n",
    "   $$\n",
    "   z_1 = \\frac{1}{1 + e^{-v_1}} = \\frac{1}{1 + e^{-0.166925}} = 0.541635\n",
    "   $$\n",
    "\n",
    "#### **Error Calculation**\n",
    "\n",
    "Calculate the error for the first input:\n",
    "$$\n",
    "E_1 = (z_1 - t_1)^2 = (0.541635 - 0.707)^2 = 0.0273457\n",
    "$$\n",
    "\n",
    "#### **Backpropagation and Weight Update**\n",
    "\n",
    "1. Compute the gradients for the output layer:\n",
    "   $$\n",
    "   p_1 = (z_1 - t_1) \\cdot z_1 \\cdot (1 - z_1) = (0.541635 - 0.707) \\cdot 0.541635 \\cdot (1 - 0.541635) = -0.041055\n",
    "   $$\n",
    "2. Compute the gradients for the hidden layer:\n",
    "   $$\n",
    "   q_1 = p_1 \\cdot b_1 \\cdot y_1 \\cdot (1 - y_1) = -0.041055 \\cdot 0.409993 \\cdot 0.613023 \\cdot (1 - 0.613023) = -0.003993\n",
    "   $$\n",
    "3. Update the weights:\n",
    "   - $\\Delta a_0 = -\\beta \\cdot q_1 = 0.000399$, \n",
    "     so $a_0' = 0.301843$\n",
    "   - $\\Delta a_1 = -\\beta \\cdot q_1 \\cdot x_1 = 0.000314$, \n",
    "     so $a_1' = 0.202268$\n",
    "   - $\\Delta b_0 = -\\beta \\cdot p_1 = 0.004105$, \n",
    "     so $b_0' = -0.080305$\n",
    "   - $\\Delta b_1 = -\\beta \\cdot p_1 \\cdot y_1 = 0.002517$, \n",
    "     so $b_1' = 0.412510$\n"
   ],
   "id": "9638638f684504fd"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-09-29T22:27:00.444162Z",
     "start_time": "2024-09-29T22:27:00.432617Z"
    }
   },
   "cell_type": "code",
   "source": [
    "# Step 1: Training with the first input (x1)\n",
    "print(\"=== Step 1: Training with First Input (x1) ===\")\n",
    "\n",
    "# Step 1.1: Forward pass for the first input\n",
    "print(\"=== Forward Pass ===\")\n",
    "u1 = a_0 + a_1 * x1\n",
    "print(f\"u1 = a_0 + a_1 * x1 = {a_0} + {a_1} * {x1} = {u1}\")\n",
    "y1 = sigmoid(u1)\n",
    "print(f\"y1 = sigmoid(u1) = sigmoid({u1}) = {y1}\")\n",
    "v1 = b_0 + b_1 * y1\n",
    "print(f\"v1 = b_0 + b_1 * y1 = {b_0} + {b_1} * {y1} = {v1}\")\n",
    "z1 = sigmoid(v1)\n",
    "print(f\"z1 = sigmoid(v1) = sigmoid({v1}) = {z1}\\n\")\n",
    "\n",
    "# Step 1.2: Error calculation for the first input\n",
    "print(\"=== Error Calculation ===\")\n",
    "error_1 = (z1 - t1) ** 2\n",
    "print(f\"Error for x1: E1 = (z1 - t1)^2 = ({z1} - {t1})^2 = {error_1}\\n\")\n",
    "\n",
    "# Step 1.3: Backpropagation - Compute gradients for the first input\n",
    "print(\"=== Backpropagation - Gradient Calculation ===\")\n",
    "p1 = (z1 - t1) * sigmoid_derivative(v1)\n",
    "print(f\"p1 (output layer gradient) = (z1 - t1) * sigmoid_derivative(v1) = ({z1} - {t1}) * sigmoid_derivative({v1}) = {p1}\")\n",
    "q1 = p1 * b_1 * sigmoid_derivative(u1)\n",
    "print(f\"q1 (hidden layer gradient) = p1 * b_1 * sigmoid_derivative(u1) = {p1} * {b_1} * sigmoid_derivative({u1}) = {q1}\\n\")\n",
    "\n",
    "# Step 1.4: Update weights based on the first input\n",
    "print(\"=== Weight Updates ===\")\n",
    "delta_b0_1 = -beta * p1\n",
    "delta_b1_1 = -beta * p1 * y1\n",
    "delta_a0_1 = -beta * q1\n",
    "delta_a1_1 = -beta * q1 * x1\n",
    "\n",
    "# New weights after the first input\n",
    "a_0_new = a_0 + delta_a0_1\n",
    "a_1_new = a_1 + delta_a1_1\n",
    "b_0_new = b_0 + delta_b0_1\n",
    "b_1_new = b_1 + delta_b1_1\n",
    "\n",
    "print(f\"Updated a_0: a_0_new = a_0 + delta_a0 = {a_0} + {delta_a0_1} = {a_0_new}\")\n",
    "print(f\"Updated a_1: a_1_new = a_1 + delta_a1 = {a_1} + {delta_a1_1} = {a_1_new}\")\n",
    "print(f\"Updated b_0: b_0_new = b_0 + delta_b0 = {b_0} + {delta_b0_1} = {b_0_new}\")\n",
    "print(f\"Updated b_1: b_1_new = b_1 + delta_b1 = {b_1} + {delta_b1_1} = {b_1_new}\\n\")\n"
   ],
   "id": "78ef9e63ebb91546",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "=== Step 1: Training with First Input (x1) ===\n",
      "=== Forward Pass ===\n",
      "u1 = a_0 + a_1 * x1 = 0.301444 + 0.201954 * 0.7853 = 0.4600384762\n",
      "y1 = sigmoid(u1) = sigmoid(0.4600384762) = 0.6130233037237685\n",
      "v1 = b_0 + b_1 * y1 = -0.0844103 + 0.409993 * 0.6130233037237685 = 0.166924963363619\n",
      "z1 = sigmoid(v1) = sigmoid(0.166924963363619) = 0.5416346103382824\n",
      "\n",
      "=== Error Calculation ===\n",
      "Error for x1: E1 = (z1 - t1)^2 = (0.5416346103382824 - 0.707)^2 = 0.02734571209797168\n",
      "\n",
      "=== Backpropagation - Gradient Calculation ===\n",
      "p1 (output layer gradient) = (z1 - t1) * sigmoid_derivative(v1) = (0.5416346103382824 - 0.707) * sigmoid_derivative(0.166924963363619) = -0.041054696305716495\n",
      "q1 (hidden layer gradient) = p1 * b_1 * sigmoid_derivative(u1) = -0.041054696305716495 * 0.409993 * sigmoid_derivative(0.4600384762) = -0.00399301629620778\n",
      "\n",
      "=== Weight Updates ===\n",
      "Updated a_0: a_0_new = a_0 + delta_a0 = 0.301444 + 0.000399301629620778 = 0.30184330162962075\n",
      "Updated a_1: a_1_new = a_1 + delta_a1 = 0.201954 + 0.00031357156974119694 = 0.20226757156974118\n",
      "Updated b_0: b_0_new = b_0 + delta_b0 = -0.0844103 + 0.00410546963057165 = -0.08030483036942834\n",
      "Updated b_1: b_1_new = b_1 + delta_b1 = 0.409993 + 0.002516748556270632 = 0.41250974855627065\n",
      "\n"
     ]
    }
   ],
   "execution_count": 4
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "## **Step 2**\n",
    "\n",
    "### **Training with Second Input $x_2$**\n",
    "\n",
    "#### **Forward Pass**\n",
    "\n",
    "1. Compute the weighted sum for the hidden layer:\n",
    "   $$\n",
    "   u_2 = a_0' + a_1' \\cdot x_2 = 0.301843 + 0.202268 \\cdot 1.57 = 0.619403\n",
    "   $$\n",
    "2. Apply the sigmoid activation to get the hidden layer output:\n",
    "   $$\n",
    "   y_2 = \\frac{1}{1 + e^{-u_2}} = \\frac{1}{1 + e^{-0.619403}} = 0.650083\n",
    "   $$\n",
    "3. Compute the weighted sum for the output layer:\n",
    "   $$\n",
    "   v_2 = b_0' + b_1' \\cdot y_2 = -0.080305 + 0.412510 \\cdot 0.650083 = 0.187861\n",
    "   $$\n",
    "4. Apply the sigmoid activation to get the network output:\n",
    "   $$\n",
    "   z_2 = \\frac{1}{1 + e^{-v_2}} = \\frac{1}{1 + e^{-0.187861}} = 0.546828\n",
    "   $$\n",
    "\n",
    "#### **Error Calculation**\n",
    "\n",
    "Calculate the error for the second input:\n",
    "$$\n",
    "E_2 = (z_2 - t_2)^2 = (0.546828 - 1.0)^2 = 0.205365\n",
    "$$\n",
    "\n",
    "#### **Backpropagation and Weight Update**\n",
    "\n",
    "1. Compute the gradients for the output layer:\n",
    "   $$\n",
    "   p_2 = (z_2 - t_2) \\cdot z_2 \\cdot (1 - z_2) = (0.546828 - 1.0) \\cdot 0.546828 \\cdot (1 - 0.546828) = -0.112299\n",
    "   $$\n",
    "2. Compute the gradients for the hidden layer:\n",
    "   $$\n",
    "   q_2 = p_2 \\cdot b_1' \\cdot y_2 \\cdot (1 - y_2) = -0.112299 \\cdot 0.412510 \\cdot 0.650083 \\cdot (1 - 0.650083) = -0.010538\n",
    "   $$\n",
    "3. Update the weights:\n",
    "   - $\\Delta a_0 = -\\beta \\cdot q_2 = 0.001054$, so $a_0'' = 0.302897$\n",
    "   - $\\Delta a_1 = -\\beta \\cdot q_2 \\cdot x_2 = 0.001654$, so $a_1'' = 0.203922$\n",
    "   - $\\Delta b_0 = -\\beta \\cdot p_2 = 0.011230$, so $b_0'' = -0.069075$\n",
    "   - $\\Delta b_1 = -\\beta \\cdot p_2 \\cdot y_2 = 0.007300$, so $b_1'' = 0.419810$"
   ],
   "id": "216b0cd9c2bf81f6"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-09-29T22:27:26.380491Z",
     "start_time": "2024-09-29T22:27:26.368248Z"
    }
   },
   "cell_type": "code",
   "source": [
    "# Step 2: Training with the second input (x2)\n",
    "print(\"=== Step 2: Training with Second Input (x2) ===\")\n",
    "\n",
    "# Step 2.1: Forward pass for the second input with updated weights\n",
    "print(\"=== Forward Pass ===\")\n",
    "u2 = a_0_new + a_1_new * x2\n",
    "print(f\"u2 = a_0_new + a_1_new * x2 = {a_0_new} + {a_1_new} * {x2} = {u2}\")\n",
    "y2 = sigmoid(u2)\n",
    "print(f\"y2 = sigmoid(u2) = sigmoid({u2}) = {y2}\")\n",
    "v2 = b_0_new + b_1_new * y2\n",
    "print(f\"v2 = b_0_new + b_1_new * y2 = {b_0_new} + {b_1_new} * {y2} = {v2}\")\n",
    "z2 = sigmoid(v2)\n",
    "print(f\"z2 = sigmoid(v2) = sigmoid({v2}) = {z2}\\n\")\n",
    "\n",
    "# Step 2.2: Error calculation for the second input\n",
    "print(\"=== Error Calculation ===\")\n",
    "error_2 = (z2 - t2) ** 2\n",
    "print(f\"Error for x2: E2 = (z2 - t2)^2 = ({z2} - {t2})^2 = {error_2}\\n\")\n",
    "\n",
    "# Step 2.3: Backpropagation - Compute gradients for the second input\n",
    "print(\"=== Backpropagation - Gradient Calculation ===\")\n",
    "p2 = (z2 - t2) * sigmoid_derivative(v2)\n",
    "print(f\"p2 (output layer gradient) = (z2 - t2) * sigmoid_derivative(v2) = ({z2} - {t2}) * sigmoid_derivative({v2}) = {p2}\")\n",
    "q2 = p2 * b_1_new * sigmoid_derivative(u2)\n",
    "print(f\"q2 (hidden layer gradient) = p2 * b_1_new * sigmoid_derivative(u2) = {p2} * {b_1_new} * sigmoid_derivative({u2}) = {q2}\\n\")\n",
    "\n",
    "# Step 2.4: Update weights based on the second input\n",
    "print(\"=== Weight Updates ===\")\n",
    "delta_b0_2 = -beta * p2\n",
    "delta_b1_2 = -beta * p2 * y2\n",
    "delta_a0_2 = -beta * q2\n",
    "delta_a1_2 = -beta * q2 * x2\n",
    "\n",
    "# Final updated weights after the second input\n",
    "a_0_final = a_0_new + delta_a0_2\n",
    "a_1_final = a_1_new + delta_a1_2\n",
    "b_0_final = b_0_new + delta_b0_2\n",
    "b_1_final = b_1_new + delta_b1_2\n",
    "\n",
    "print(f\"Updated a_0: a_0_final = a_0_new + delta_a0_2 = {a_0_new} + {delta_a0_2} = {a_0_final}\")\n",
    "print(f\"Updated a_1: a_1_final = a_1_new + delta_a1_2 = {a_1_new} + {delta_a1_2} = {a_1_final}\")\n",
    "print(f\"Updated b_0: b_0_final = b_0_new + delta_b0_2 = {b_0_new} + {delta_b0_2} = {b_0_final}\")\n",
    "print(f\"Updated b_1: b_1_final = b_1_new + delta_b1_2 = {b_1_new} + {delta_b1_2} = {b_1_final}\\n\")\n"
   ],
   "id": "a363c133c4edf9c7",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "=== Step 2: Training with Second Input (x2) ===\n",
      "=== Forward Pass ===\n",
      "u2 = a_0_new + a_1_new * x2 = 0.30184330162962075 + 0.20226757156974118 * 1.57 = 0.6194033889941144\n",
      "y2 = sigmoid(u2) = sigmoid(0.6194033889941144) = 0.6500828465571634\n",
      "v2 = b_0_new + b_1_new * y2 = -0.08030483036942834 + 0.41250974855627065 * 0.6500828465571634 = 0.18786068120461183\n",
      "z2 = sigmoid(v2) = sigmoid(0.18786068120461183) = 0.5468275328862457\n",
      "\n",
      "=== Error Calculation ===\n",
      "Error for x2: E2 = (z2 - t2)^2 = (0.5468275328862457 - 1.0)^2 = 0.20536528494996667\n",
      "\n",
      "=== Backpropagation - Gradient Calculation ===\n",
      "p2 (output layer gradient) = (z2 - t2) * sigmoid_derivative(v2) = (0.5468275328862457 - 1.0) * sigmoid_derivative(0.18786068120461183) = -0.11229939210967113\n",
      "q2 (hidden layer gradient) = p2 * b_1_new * sigmoid_derivative(u2) = -0.11229939210967113 * 0.41250974855627065 * sigmoid_derivative(0.6194033889941144) = -0.01053769346760773\n",
      "\n",
      "=== Weight Updates ===\n",
      "Updated a_0: a_0_final = a_0_new + delta_a0_2 = 0.30184330162962075 + 0.001053769346760773 = 0.3028970709763815\n",
      "Updated a_1: a_1_final = a_1_new + delta_a1_2 = 0.20226757156974118 + 0.0016544178744144137 = 0.2039219894441556\n",
      "Updated b_0: b_0_final = b_0_new + delta_b0_2 = -0.08030483036942834 + 0.011229939210967113 = -0.06907489115846123\n",
      "Updated b_1: b_1_final = b_1_new + delta_b1_2 = 0.41250974855627065 + 0.007300390848929406 = 0.41981013940520007\n",
      "\n"
     ]
    }
   ],
   "execution_count": 5
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "## **Step 3**\n",
    "\n",
    "### **Part 1(ii): Network Output and MSE Calculation for Both Inputs**\n",
    "\n",
    "After updating the weights for both inputs, we calculate the final network outputs and the Mean Square Error (MSE).\n",
    "\n",
    "#### **Final Forward Pass for $x_1$ (Recalculating $z_1'$)**\n",
    "\n",
    "1. **Compute the weighted sum for the hidden layer**:\n",
    "   $$\n",
    "   u_1' = a_0'' + a_1'' \\cdot x_1 = 0.302897 + 0.203922 \\cdot 0.7853 = 0.463467\n",
    "   $$\n",
    "2. **Compute the hidden layer output**:\n",
    "   $$\n",
    "   y_1' = \\frac{1}{1 + e^{-u_1'}} = \\frac{1}{1 + e^{-0.463467}} = 0.613857\n",
    "   $$\n",
    "3. **Compute the weighted sum for the output layer**:\n",
    "   $$\n",
    "   v_1' = b_0'' + b_1'' \\cdot y_1' = -0.069075 + 0.419810 \\cdot 0.613857 = 0.188855\n",
    "   $$\n",
    "4. **Compute the output for $x_1$**:\n",
    "   $$\n",
    "   z_1' = \\frac{1}{1 + e^{-v_1'}} = \\frac{1}{1 + e^{-0.188855}} = 0.547005\n",
    "   $$\n",
    "5. **Final error for $x_1$**:\n",
    "   $$\n",
    "   E_1' = (z_1' - t_1)^2 = (0.547005 - 0.707)^2 = 0.025598\n",
    "   $$\n",
    "\n",
    "#### **Final Forward Pass for $x_2$ (Recalculating $z_2'$)**\n",
    "\n",
    "1. **Compute the weighted sum for the hidden layer**:\n",
    "   $$\n",
    "   u_2' = a_0'' + a_1'' \\cdot x_2 = 0.302897 + 0.203922 \\cdot 1.57 = 0.622052\n",
    "   $$\n",
    "2. **Compute the hidden layer output**:\n",
    "   $$\n",
    "   y_2' = \\frac{1}{1 + e^{-u_2'}} = \\frac{1}{1 + e^{-0.622052}} = 0.650717\n",
    "   $$\n",
    "3. **Compute the weighted sum for the output layer**:\n",
    "   $$\n",
    "   v_2' = b_0'' + b_1'' \\cdot y_2' = -0.069075 + 0.419810 \\cdot 0.650717 = 0.204835\n",
    "   $$\n",
    "4. **Compute the output for $x_2$**:\n",
    "   $$\n",
    "   z_2' = \\frac{1}{1 + e^{-v_2'}} = \\frac{1}{1 + e^{-0.204835}} = 0.550870\n",
    "   $$\n",
    "5. **Final error for $x_2$**:\n",
    "   $$\n",
    "   E_2' = (z_2' - t_2)^2 = (0.550870 - 1.0)^2 = 0.201718\n",
    "   $$\n",
    "\n",
    "#### **Mean Square Error (MSE)**\n",
    "\n",
    "Finally, we calculate the Mean Square Error (MSE) as the average of the final errors for both inputs:\n",
    "$$\n",
    "\\text{MSE} = \\frac{E_1' + E_2'}{2} = \\frac{0.025598 + 0.201718}{2} = 0.113658\n",
    "$$"
   ],
   "id": "4c2a6dcfec36ad07"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-09-29T22:27:29.934792Z",
     "start_time": "2024-09-29T22:27:29.923947Z"
    }
   },
   "cell_type": "code",
   "source": [
    "# Step 3: MSE Calculation for both inputs using final weights\n",
    "print(\"=== Step 3: MSE Calculation ===\")\n",
    "\n",
    "# Final forward pass for the first input\n",
    "u1_final = a_0_final + a_1_final * x1\n",
    "y1_final = sigmoid(u1_final)\n",
    "v1_final = b_0_final + b_1_final * y1_final\n",
    "z1_final = sigmoid(v1_final)\n",
    "error_1_final = (z1_final - t1) ** 2\n",
    "print(f\"Final output for x1: z1_final = {z1_final}, Final error for x1: E1_final = {error_1_final}\")\n",
    "\n",
    "# Final forward pass for the second input\n",
    "u2_final = a_0_final + a_1_final * x2\n",
    "y2_final = sigmoid(u2_final)\n",
    "v2_final = b_0_final + b_1_final * y2_final\n",
    "z2_final = sigmoid(v2_final)\n",
    "error_2_final = (z2_final - t2) ** 2\n",
    "print(f\"Final output for x2: z2_final = {z2_final}, Final error for x2: E2_final = {error_2_final}\\n\")\n",
    "\n",
    "# MSE calculation\n",
    "mse = (error_1_final + error_2_final) / 2\n",
    "print(f\"Mean Square Error (MSE) = ({error_1_final} + {error_2_final}) / 2 = {mse}\")\n"
   ],
   "id": "abafe498ccced904",
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "=== Step 3: MSE Calculation ===\n",
      "Final output for x1: z1_final = 0.5470050431163832, Final error for x1: E1_final = 0.025598386228190384\n",
      "Final output for x2: z2_final = 0.5508696279584414, Final error for x2: E2_final = 0.20171809109018884\n",
      "\n",
      "Mean Square Error (MSE) = (0.025598386228190384 + 0.20171809109018884) / 2 = 0.11365823865918961\n"
     ]
    }
   ],
   "execution_count": 6
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "## **Conclusion**\n",
    "\n",
    "### **Summary of Results:**\n",
    "\n",
    "1. **Part 1(i)**:\n",
    "   - Initial output for $x_1 = 0.7853$: $z_1 = 0.541635$\n",
    "   - Error for $x_1$: $E_1 = 0.0273457$\n",
    "   - Updated weights after training on $x_1$:\n",
    "     - $a_0' = 0.301843$\n",
    "     - $a_1' = 0.202268$\n",
    "     - $b_0' = -0.080305$\n",
    "     - $b_1' = 0.412510$\n",
    "\n",
    "2. **Part 1(ii)**:\n",
    "   - Initial output for $x_2 = 1.57$: $z_2 = 0.546828$\n",
    "   - Error for $x_2$: $E_2 = 0.205365$\n",
    "   - Final output for $x_1$ after weight updates: $z_1' = 0.547005$\n",
    "   - Final output for $x_2$ after weight updates: $z_2' = 0.550870$\n",
    "   - Mean Square Error (MSE): $0.1137$\n",
    "\n",
    "The network was successfully trained using **example-by-example training**, and the final Mean Square Error (MSE) after training on both inputs was **0.1137**."
   ],
   "id": "31852503e56eb99b"
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}