sdk/python/using-mlflow/train-and-log/xgboost_service_principal.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Training and tracking an XGBoost classifier with MLflow using Service Principal Authentication\n",
    "\n",
    "This notebook demonstrates how to use MLflow for tracking experiment using MLflow in Azure ML using a Service Principal to authenticate against Azure. Authentication is automatically handled for you when you are running inside an Azure ML compute (Compute Instances, Compute Clusters). However, if you are using any other type of compute (like Azure Databricks, your laptop, etc) then you will have to provide credentials to be able to access Azure ML Services. By default, the `azureml-mlflow` plug-in uses Interactive Authentication. However, they may be cases where you are not able to use interactivity - for instance when you are running inside of a job or unattended system). On those cases, you can use a Service Principal to authenticate against the services. On this example, we will walk you through the steps to train a model using MLflow connected to Azure ML using a Service Principal.\n",
    "\n",
    "We will consider the [Heart Disease Data Set](https://archive.ics.uci.edu/ml/datasets/heart+disease). This database contains 76 attributes, but we will be using a subset of 14 of them. The \"goal\" field refers to the presence of heart disease in the patient. It is integer valued from 0 (no presence) to 4. In this example we will concentrated on simply attempting to distinguish presence (values 1,2,3,4) from absence (value 0)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Ensure you have the dependencies for this notebook\n",
    "%pip install -r xgboost_service_principal.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import warnings\n",
    "\n",
    "warnings.simplefilter(\"ignore\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Configuring the experiment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Configuring how MLflow will authenticate\n",
    "\n",
    "We are going to configure the communication between Azure ML and MLflow. By default, the plug-in `azureml-mlflow` uses interactive authentication to authenticate against Azure. However, we can change this by populating specific environment variables. The following environment variables, if configured, will result in the communication being established using a Service Principal:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "os.environ[\"AZURE_TENANT_ID\"] = \"<AZURE_TENANT_ID>\"\n",
    "os.environ[\"AZURE_CLIENT_ID\"] = \"<AZURE_CLIENT_ID>\"\n",
    "os.environ[\"AZURE_CLIENT_SECRET\"] = \"<AZURE_CLIENT_SECRET>\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Getting access to the workspace\n",
    "\n",
    "We can use this credentials to get access to the workspace. To do that we will need the workspace details:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "subscription_id = \"<SUBSCRIPTION_ID>\"\n",
    "resource_group = \"<RESOURCE_GROUP>\"\n",
    "workspace = \"<AML_WORKSPACE_NAME>\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azure.ai.ml import MLClient\n",
    "from azure.identity import DefaultAzureCredential"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> **Hint for automation:** If you are running inside a context where Azure CLI is installed, like a GitHub Workflow or an Azure DevOps Pipeline, you can use here `AzureCliCredential` instead of `DefaultAzureCredential` to get the associated credentials. In that case, configuring the environment variables `AZURE_TENANT_ID`, `AZURE_CLIENT_ID` and `AZURE_CLIENT_SECRET` is not required."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "credentials = DefaultAzureCredential()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We could have used `ClientSecretCredential` instead of `DefaultAzureCredential` and pass `client_id`, `client_secret` and `tenant_id`. However, since we already configured the environment variables, we can skip that step and use `DefaultAzureCredential` which will pick up the values for us."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ml_client = MLClient(\n",
    "    subscription_id=subscription_id,\n",
    "    resource_group_name=resource_group,\n",
    "    credential=credentials,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ws = ml_client.workspaces.get(name=workspace)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Configuring tracking URL in Mflow and creating the experiment\n",
    "\n",
    "We can now use `ws.mlflow_tracking_uri` to get access to the tracking URL for the given workspace. Since the environment variables we configured before are populated, when we call `mlflow.set_experiment`, those credentials will be used to authenticate against the service."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import mlflow\n",
    "\n",
    "mlflow.set_tracking_uri = ws.mlflow_tracking_uri\n",
    "mlflow.set_experiment(experiment_name=\"heart-condition-classifier\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Exploring the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "file_url = \"https://azuremlexampledata.blob.core.windows.net/data/heart-disease-uci/data/heart.csv\"\n",
    "df = pd.read_csv(file_url)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see, some of the variables are categorical. To make it simpler for our model to handle these values, let's use their encoded values:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df[\"thal\"] = df[\"thal\"].astype(\"category\").cat.codes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The encoded values looks then as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df[\"thal\"].unique()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's split our dataset in train and test, so we can assess the performance of the model without overfitting the dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(\n",
    "    df.drop(\"target\", axis=1), df[\"target\"], test_size=0.3\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training a model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We are going to use autologging capabilities in MLflow to track parameters and metrics:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mlflow.xgboost.autolog()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's create a simple classifier and train it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from xgboost import XGBClassifier\n",
    "\n",
    "model = XGBClassifier(use_label_encoder=False, eval_metric=\"logloss\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As soon as the `train` method is executed, MLflow will stat a run in Azure ML to start tracking the experiment's run. However, it is always a good idea to start the run manually so you have the run ID at hand quickly. This is not required though.\n",
    "\n",
    "> Important: When running training routines in Azure ML as jobs, you don't need to start or end the run in your training code as it is automatically done for you by Azure ML."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "run = mlflow.start_run()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.fit(X_train, y_train, eval_set=[(X_test, y_test)], verbose=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Logging extra metrics\n",
    "\n",
    "Autolog capabilities in XGBoost will log metrics like validation loss, however, it won't log any specific metric in a classification problem. In this case, we are going to pay closer attention to our ability to detect heart condition while avoiding a type II error as much as possible. To calculate the metric, we are going to use our test dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred = model.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import accuracy_score, recall_score\n",
    "\n",
    "accuracy = accuracy_score(y_test, y_pred)\n",
    "recall = recall_score(y_test, y_pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Accuracy: %.2f%%\" % (accuracy * 100.0))\n",
    "print(\"Recall: %.2f%%\" % (recall * 100.0))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exploring the expriment with MLFlow\n",
    "\n",
    "Let's first end the experiment run so we can review it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mlflow.end_run()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can query the run again to see what's been logged:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "run = mlflow.get_run(run.info.run_id)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's explore the parameters that got logged:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pd.DataFrame(data=[run.data.params], index=[\"Value\"]).T"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's explore the metrics' values:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pd.DataFrame(data=[run.data.metrics], index=[\"Value\"]).T"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Pay attention how metrics calculated with `scikit-learn` were automatically tracked for us. None of them were manually added to the run. Also, MLflow uses naming conventions including the variable's names to help understand what was logged. `X_test` was added to the name of the metric meaning that it corresponds to the metric in the testing split of the dataset."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's explore artifacts that got logged in the run. This requires us to use the MLflow client:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "client = mlflow.tracking.MlflowClient()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "client.list_artifacts(run_id=run.info.run_id)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see in this example, three artifacts are availble in the run:\n",
    "\n",
    "* `feature_importance_weight.json` -> the feature importance of the model we created.\n",
    "* `feature_importance_weight.png` -> a plot of the feature importance mentioned above, stored as an image.\n",
    "* `metric_info.json` -> contains a json representation of all the metrics captured by the XGBoost.\n",
    "* `model`, the path where the model is stored. Note that this artifact is a directory.\n",
    "\n",
    "You can download any artifact using the method `download_artifact`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "file_path = mlflow.artifacts.download_artifacts(\n",
    "    run_id=run.info.run_id, artifact_path=\"feature_importance_weight.png\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since the artifact is an image, we can display it in the following way:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import matplotlib.image as img\n",
    "\n",
    "image = img.imread(file_path)\n",
    "plt.imshow(image)\n",
    "plt.show()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.10 - SDK V2",
   "language": "python",
   "name": "python310-sdkv2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.4 (main, Mar 31 2022, 08:41:55) [GCC 7.5.0]"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}