supervised.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "TYjuU2jkM95v"
   },
   "source": [
    "# Supervised Learning Tutorial: Heart Disease Prediction\n",
    "\n",
    "This tutorial demonstrates how to use different supervised learning models in Python. We'll cover:\n",
    "\n",
    "- **Multilayer Perceptrons (MLP)**\n",
    "- **Decision Trees and Random Forests**\n",
    "- **Support Vector Machines (SVM)**\n",
    "- **Naïve Bayes**\n",
    "\n",
    "The dataset contains patient health indicators for predicting heart disease."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "mRX78-N_NCVU"
   },
   "source": [
    "## **Step 1: Load and Inspect the Data**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "qxV-lYVjMKWh"
   },
   "source": [
    "# Prepare the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "htLYigP4MMKW"
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "# Load the data\n",
    "# Replace with your file path if needed\n",
    "file_path = 'heartattack.csv'\n",
    "data = pd.read_csv(file_path)\n",
    "\n",
    "# Inspect the first few rows\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "jkmkF57ZNYpL"
   },
   "source": [
    "## **Step 2: Data Preparation**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "UMkEV_tWNvb8"
   },
   "outputs": [],
   "source": [
    "# NOTE: we have already split the train and test\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "# Separate features and target\n",
    "X = data.drop('output', axis=1)\n",
    "y = data['output']\n",
    "\n",
    "# Split into train and test sets\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "Q4UGfDF3N0gw"
   },
   "outputs": [],
   "source": [
    "# Directly load the train and test data\n",
    "data_train = pd.read_csv('train.csv')\n",
    "data_test = pd.read_csv('test.csv')\n",
    "\n",
    "X_train = data_train.drop('output', axis=1)\n",
    "y_train = data_train['output']\n",
    "X_test = data_test.drop('output', axis=1)\n",
    "y_test = data_test['output']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "XQmrtNosNYCB"
   },
   "outputs": [],
   "source": [
    "# NOTE: Feature engineering is optional!\n",
    "# But it always good to do \"certain\" preprocessing\n",
    "# Add your own \"inductive bias\" into the data\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "# Standardize the features\n",
    "scaler = StandardScaler()\n",
    "X_train = scaler.fit_transform(X_train)\n",
    "X_test = scaler.transform(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "c9kUHp67OODn"
   },
   "source": [
    "## **Step 3: Multilayer Perceptron (MLP) Classifier**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "e7cwsTuUOQIL"
   },
   "outputs": [],
   "source": [
    "from sklearn.neural_network import MLPClassifier\n",
    "from sklearn.metrics import accuracy_score\n",
    "\n",
    "# Train the MLP model\n",
    "mlp = MLPClassifier(hidden_layer_sizes=(100,), max_iter=500, random_state=42)\n",
    "mlp.fit(X_train, y_train)\n",
    "\n",
    "# Evaluate the model\n",
    "y_pred = mlp.predict(X_test)\n",
    "print(\"MLP Classifier Accuracy:\", accuracy_score(y_test, y_pred))\n",
    "\n",
    "# There are other python packages you could use for training an MLP or neural network, e.g., `pytorch`, `tensorflow`, `jax`. \n",
    "# They are more flexible and powerful, but also more complex to use -> they have an auto-differentiation feature that allows you to define your own loss function and optimization algorithm."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "MNVoHjjEOVPu"
   },
   "source": [
    "**Changeable Parameters:**\n",
    "- `hidden_layer_sizes`: Adjust the network size (e.g., `(50,)`, `(100, 50)`)\n",
    "- `max_iter`: Increase if the model is underfitting."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "XcuWIm7IOYgw"
   },
   "source": [
    "## **Step 4: Decision Trees and Random Forests**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "EvLwsYlmObff"
   },
   "outputs": [],
   "source": [
    "from sklearn.tree import DecisionTreeClassifier\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "\n",
    "# Decision Tree\n",
    "dt = DecisionTreeClassifier(max_depth=5, random_state=42)\n",
    "dt.fit(X_train, y_train)\n",
    "dt_pred = dt.predict(X_test)\n",
    "print(\"Decision Tree Accuracy:\", accuracy_score(y_test, dt_pred))\n",
    "\n",
    "# Random Forest\n",
    "rf = RandomForestClassifier(n_estimators=100, random_state=42)\n",
    "rf.fit(X_train, y_train)\n",
    "rf_pred = rf.predict(X_test)\n",
    "print(\"Random Forest Accuracy:\", accuracy_score(y_test, rf_pred))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "XhMkp8t_OjPg"
   },
   "source": [
    "**Changeable Parameters:**\n",
    "- `max_depth` for Decision Trees\n",
    "- `n_estimators` for Random Forests"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "EuRJhyZ1OlaM"
   },
   "source": [
    "## **Step 5: Support Vector Machines (SVM)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "zidoYS9yOnsC"
   },
   "outputs": [],
   "source": [
    "from sklearn.svm import SVC\n",
    "\n",
    "# Train the SVM model\n",
    "svm = SVC(kernel='rbf', C=1.0, random_state=42)\n",
    "svm.fit(X_train, y_train)\n",
    "\n",
    "# Evaluate the model\n",
    "svm_pred = svm.predict(X_test)\n",
    "print(\"SVM Accuracy:\", accuracy_score(y_test, svm_pred))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8Y2v1BLqO0SQ"
   },
   "source": [
    "**Changeable Parameters:**\n",
    "- `kernel`: Try 'linear', 'poly', 'sigmoid'\n",
    "- `C`: Adjust for more or less regularization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "AryEdrOtO8OV"
   },
   "source": [
    "## **Step 6: Naïve Bayes**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "grYUr9mhO91k"
   },
   "outputs": [],
   "source": [
    "from sklearn.naive_bayes import GaussianNB\n",
    "\n",
    "# Train the Naïve Bayes model\n",
    "nb = GaussianNB()\n",
    "nb.fit(X_train, y_train)\n",
    "\n",
    "# Evaluate the model\n",
    "nb_pred = nb.predict(X_test)\n",
    "print(\"Naïve Bayes Accuracy:\", accuracy_score(y_test, nb_pred))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5UUt6SCTO_r5"
   },
   "source": [
    "**Note:** Naïve Bayes has few parameters to adjust but works well for certain types of problems."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "uLkC5an5PCX9"
   },
   "source": [
    "**Next Steps:**\n",
    "- Try tuning hyperparameters for each model.\n",
    "- Explore cross-validation.\n",
    "- Add visualization of results if needed.\n",
    "\n",
    "Happy coding!"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "name": "python3"
  },
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}