02_bikes_demand.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Cycling Demand"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![imgs](images/bikes_in_street.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Table of Contents"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. Overview\n",
    "2. Dependencies\n",
    "3. Data\n",
    "4. Model Training\n",
    "5. Model Evaluation\n",
    "6. Summary\n",
    "7. Exercises"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Overview"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When we travel, matter where you end up around the globe, chances are that we will end up \n",
    "stopping by a city with a bikes sharing program in place. I, personally, have had the privilege \n",
    "of renting bikes by the minute in Washington DC, New York City, London, Lyon, and a few other, \n",
    "places. But, how do the governments or companies in charge of this kind of service \n",
    "- figure out where a docking station should be located at? or,\n",
    "- how many bikes should each station have? or,\n",
    "- how quickly will they be rented and/or replenished?\n",
    "- ...\n",
    "\n",
    "All of these are important questions, and if the person or team in charge of figuring out the answers \n",
    "is using machine learning to figure this out, chances are, a \"put 10 bikes here and 20 there, trust me \n",
    "the algorithm says it\" won't suffice. Context, and sometimes persuasive arguments, can go a long way.\n",
    "\n",
    "In this example, we will explain the behavior of a regression model on the Bike rentals[1] dataset. We will show how to calculate the partial dependence (PD) and the individual conditional expectation (ICE) to determine the feature effects on the model.\n",
    "\n",
    "We will follow the example from the PDP chapter of the Interpretable Machine Learning[2] book and use the cleaned version of the dataset from the github repository."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Dependencies"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here are the packages we will be using in this notebook.\n",
    "\n",
    "- `scikit-learn`\n",
    "- `pandas`\n",
    "- `joblib`\n",
    "- `matplotlib`\n",
    "- `alibi`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install scikit-learn pandas joblib matplotlib alibi numpy rich"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from alibi.explainers import PartialDependence, plot_pd\n",
    "from sklearn.preprocessing import StandardScaler, OneHotEncoder, OrdinalEncoder\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.ensemble import RandomForestRegressor\n",
    "from sklearn.compose import ColumnTransformer\n",
    "from sklearn.pipeline import Pipeline\n",
    "from os.path import join\n",
    "import pandas as pd\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will be using an adapted version of the [bikes sharing dataset from Washington DC](http://archive.ics.uci.edu/dataset/275/bike+sharing+dataset). This version was taken from Christoph Molnar's excellent book, Interpretable Machine Learning, and this \n",
    "notebook was written by some of my wonderful colleagues and adapted by me. \n",
    "\n",
    "The variables include:\n",
    "- season\n",
    "- yr\n",
    "- mnth\n",
    "- holiday\n",
    "- weekday\n",
    "- workingday\n",
    "- weathersit\n",
    "- temp\n",
    "- hum\n",
    "- windspeed\n",
    "- cnt\n",
    "- days_since_2011"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv('https://raw.githubusercontent.com/christophM/interpretable-ml-book/master/data/bike.csv')\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will be doing a bit of work (massaging) the features before creating a model so let's get their names first."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "feature_names = df.columns.tolist()\n",
    "feature_names.remove('cnt')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We need to separate the categoricals from the numerical to add the transformation process to a \n",
    "well-defined pipeline. In addition, we will also need the index of each."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "categorical_columns_names   = ['season', 'yr', 'mnth', 'holiday', 'weekday', 'workingday', 'weathersit']\n",
    "categorical_columns_indices = [feature_names.index(cn) for cn in categorical_columns_names]\n",
    "numerical_columns_indices   = [feature_names.index(fn) for fn in feature_names if fn not in categorical_columns_names]\n",
    "\n",
    "categorical_columns_indices, numerical_columns_indices"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Time to split the data. Even though you will see 0.20 below, feel free to switch the number to whichever split you'd like."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(\n",
    "    df[feature_names], df['cnt'], test_size=0.2, random_state=0\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We need to change the string or object columns like season and holiday into ordinal variables \n",
    "to make them easier to work with our algorithms. Let's do that with the help of the \n",
    "`OrdinalEncoder()` class from `scikit-learn`. After we fit our training set to it we can transform \n",
    "both the training and the test set in the same fashion."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "oe = OrdinalEncoder().fit(X_train[categorical_columns_names])\n",
    "oe"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here are the categories taken by our object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "categorical_names = {i: list(v) for (i, v) in zip(categorical_columns_indices, oe.categories_)}\n",
    "categorical_names"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train.loc[:, categorical_columns_names] = oe.transform(X_train[categorical_columns_names])\n",
    "X_test.loc[:, categorical_columns_names]  = oe.transform(X_test[categorical_columns_names])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train[categorical_columns_names].head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "oe.categories_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we can to also create dummy variables from our categorical data. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cat_transf = OneHotEncoder(\n",
    "    categories=[range(len(x)) for x in categorical_names.values()],\n",
    "    handle_unknown='ignore',\n",
    ")\n",
    "cat_transf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lastly, our numerical values will be scaled using the `StandardScaler` class of `scikit-learn`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# define numerical standard sclaer\n",
    "num_transf = StandardScaler()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now create a `ColumnTransformer` pipeline with the one-hot encoder and the standard scaler \n",
    "classes so that we can transform new inputs on the fly as they arrive."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "preprocessor = ColumnTransformer(\n",
    "    transformers=[\n",
    "        ('cat', cat_transf, categorical_columns_indices),\n",
    "        ('num', num_transf, numerical_columns_indices),\n",
    "    ],\n",
    "    sparse_threshold=0\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# fit preprocessor\n",
    "preprocessor.fit(X_train.values)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train_ohe = preprocessor.transform(X_train.values)\n",
    "X_test_ohe  = preprocessor.transform(X_test.values)\n",
    "X_train_ohe.shape, X_train_ohe[:2, :]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Time to train our model. 😎"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Model Training"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For our model, we will be using Random Forests.\n",
    "\n",
    "> Random forests are an ensemble machine learning method used for classification and regression.\n",
    "\n",
    "If you have never used random forests, here's an analogy that might help. Imagine you're consulting \n",
    "a crowd to predict tomorrow's weather. Asking one person might give you a bad prediction. But asking \n",
    "many people and combining their opinions gives a better result overall, and this is the idea behind random forests:\n",
    "\n",
    "1. Build multiple decision trees on random subsets of the data. \n",
    "2. Make each tree vote on the outcome and take the most popular vote.\n",
    "\n",
    "    It's similar to asking 100 randomly chosen people to each make a weather prediction based on limited \n",
    "    info. Then combining all their votes to get the consensus. More specifically, each decision tree is \n",
    "    trained on a random sample of features and data points. This introduces variance into the trees, so \n",
    "    they make somewhat different predictions. We could think of each person in the crowd having access to \n",
    "    slightly different weather data before making their prediction. \n",
    "\n",
    "3. Finally, all the trees vote and the random forest outputs the majority voted class as the prediction.\n",
    "\n",
    "This ensemble approach reduces overfitting compared to using just one decision tree. It's unlikely 100 \n",
    "people would all be wrong in the same way. The crowd wisdom improves predictions. In essence, random forests \n",
    "combine multiple decision trees to deliver accurate, robust predictions like a wise crowd. The randomness \n",
    "reduces overfitting and captures more of the complexity in the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor = RandomForestRegressor(random_state=0)\n",
    "predictor.fit(X_train_ohe, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print('Train score: %.2f' % (predictor.score(X_train_ohe, y_train)))\n",
    "print('Test score: %.2f'  % (predictor.score(X_test_ohe,  y_test)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pipe = Pipeline([('preprocess', preprocessor), ('model', predictor)])\n",
    "pipe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import joblib"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_path = 'models/bike_sharing/rf_model.pkl'\n",
    "joblib.dump(pipe, model_path)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pipe = joblib.load(model_path)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pipe.predict(X_train.values)[:50]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. Model Evaluation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The method we will evaluate in the section is called partial dependence plots.\n",
    "\n",
    "> Partial dependence plots in machine learning are like snapshots of how one specific feature \n",
    "affects a model's predictions. Imagine you have a car and want to know how its speed impacts \n",
    "fuel efficiency. A partial dependence plot shows how changing the car's speed (while keeping \n",
    "everything else constant) affects its gas mileage, helping you understand the relationship \n",
    "between speed and fuel efficiency.\n",
    "\n",
    "Imagine you have a model that predicts house prices based on size, location, etc. You \n",
    "want to know - how does size affect the predicted price?\n",
    "\n",
    "1. Fix all the other features except size. So keep location, number of bedrooms, etc. the same. \n",
    "2. Feed the model different values for size only, keeping everything else fixed. Record \n",
    "the predicted price for each size value.\n",
    "\n",
    "    This is like asking: \"If I only change the size, how would that impact the predicted price?\"\n",
    "\n",
    "3.  Plot the size values versus the predicted prices. The curve shows the partial \n",
    "dependence of price on size.\n",
    "\n",
    "An analogy is baking a cake. To understand how egg amount affects cake taste:\n",
    "\n",
    "1. Fix the flour, sugar, etc. amounts \n",
    "2. Bake cakes varying only the egg amount\n",
    "3. Plot taste vs. egg amount \n",
    "\n",
    "The curve illustrates the partial dependence of taste on eggs specifically.\n",
    "\n",
    "So in essence, partial dependence isolates one feature to study its marginal effect \n",
    "on predictions. By ignoring interactions with other features, we can better understand \n",
    "each feature's individual relationship with the target.\n",
    "\n",
    "Let's get started with partial dependance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "explainer = PartialDependence(\n",
    "    predictor=pipe.predict, feature_names=feature_names, \n",
    "    target_names=['Number of bikes'], categorical_names=categorical_names\n",
    ")\n",
    "explainer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's pick a few features of interest. Feel free to change and experiment withe the ones below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "features = [\n",
    "    feature_names.index('temp'),      feature_names.index('hum'), \n",
    "    feature_names.index('windspeed'), feature_names.index('season')\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# compute explanations\n",
    "exp = explainer.explain(\n",
    "    X=X_train.values, features=features, kind='average'\n",
    ")\n",
    "exp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_pd(\n",
    "    exp=exp, n_cols=2, sharey='row', fig_kw={'figheight': 10, 'figwidth': 15}\n",
    ");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A limitation of PDPs is that they show the average marginal effect of a feature on the predicted \n",
    "response. But this could hide interactions that cause the effect of the feature to differ across \n",
    "individual data points. We can use the Individual Conditional Expectation sampling, or ICE, to \n",
    "fix this.\n",
    "\n",
    "> ICE (Individual Conditional Expectation) sampling is a technique used with partial dependence \n",
    "plots (PDPs) to help account for interaction effects between features. It aims to uncover these \n",
    "interactions by creating one PDP per data point, conditional on the values of the other \n",
    "features for that individual point."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "exp = explainer.explain(\n",
    "    X=X_train.values, features=features, kind='both'\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The explainer above works as follows:\n",
    "\n",
    "1. We take a single row of data\n",
    "2. Then vary the feature of interest, say, `windspeed`, keeping the other features fixed at that row's values \n",
    "3. Record the prediction at each value \n",
    "4. Repeat for many rows\n",
    "5. Average the conditional PDPs across rows\n",
    "\n",
    "Now the PDP shows the isolated effect of the feature for specific combinations of the other features. Interactions \n",
    "and heterogeneity can be uncovered. For example, the effect of size on home price may differ strongly based on location. ICE sampling creates PDPs conditional on location to reveal these interactions.\n",
    "\n",
    "Let's add a seed for the ICE sampling."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.random.seed(13)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_pd(\n",
    "    exp=exp, n_cols=2, n_ice=70, sharey='row', \n",
    "    center=True, fig_kw={'figheight': 10, 'figwidth': 15}\n",
    ");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What if we wanted to observe two features at the same time to evaluate their combined effect on \n",
    "target variable while holding everything else constant? This is doable with alibi so let's pick \n",
    "some features and explain them visually."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "features = [\n",
    "    (feature_names.index('temp'), feature_names.index('windspeed')),\n",
    "    (feature_names.index('mnth'), feature_names.index('weathersit')),\n",
    "    (feature_names.index('season'), feature_names.index('temp'))\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "exp = explainer.explain(\n",
    "    X=X_train.values, features=features, kind='average', grid_resolution=25\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_pd(\n",
    "    exp=exp, n_cols=1, fig_kw={'figheight': 10, 'figwidth': 10}\n",
    ");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 8. Conclusion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. Partial dependence plots are visualizations that help us understand how a single feature in a \n",
    "machine learning model influences its predictions, holding all other features constant. In a bike \n",
    "sharing scenario, it could show how temperature affects the number of bike rentals while keeping \n",
    "other factors like humidity and day of the week fixed.\n",
    "2. Using Alibi Explain or plain matplotlib, we can create partial dependence plots that showcase \n",
    "the impact of individual features on model predictions, providing insights into how changes in \n",
    "specific variables affect bike rental predictions.\n",
    "3. ICE (Individual Conditional Expectation) sampling is a technique used in conjunction with partial \n",
    "dependence plots to create multiple plots for different instances or samples from the dataset, \n",
    "offering a more comprehensive view of feature interactions and model behavior.\n",
    "4. We need partial dependence plots to gain transparency into our machine learning models. In bike \n",
    "sharing or similar applications, where understanding how factors like weather or time affect rental \n",
    "demand, it is important to visually inspect the outputs of our models for operational decision-making \n",
    "and model trustworthiness."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 9. Exercises"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Inspecting Feature Interactions on Medical Cost Data\n",
    "\n",
    "- Load the [medical cost prediction dataset](https://www.kaggle.com/datasets/mirichoi0218/insurance).\n",
    "- Train a regression model to predict costs. \n",
    "- Use Alibi's Partial Dependance explainer, or another one of your choosing, to observe the \n",
    "interactions between different variables and the target variable. \n",
    "- Use ICE sampling to see where the average prediction lands after changing the value of your chose features.\n",
    "\n",
    "The key is tying to build intuition for examining the models. ৻(  •̀ ᗜ •́  ৻)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "xai",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}