diabetes-classification-model-training.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Training Models\n",
    "\n",
    "The central goal of machine learning is to train predictive models that can be used by applications. In Azure Machine Learning,  you can use scripts to train models leveraging common machine learning frameworks like Scikit-Learn, Tensorflow, PyTorch, SparkML, and others. You can run these training scripts as experiments in order to track metrics and outputs - in particular, the trained models.\n",
    "\n",
    "\n",
    "\n",
    "## Connect to Your Workspace\n",
    "\n",
    "The first thing you need to do is to connect to your workspace using the Azure ML SDK.\n",
    "\n",
    "> **Note**: If you do not have a current authenticated session with your Azure subscription, you'll be prompted to authenticate. Follow the instructions to authenticate using the code provided."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Ready to use Azure ML 1.40.0 to work with azureml-workspace\n"
     ]
    }
   ],
   "source": [
    "import azureml.core\n",
    "from azureml.core import Workspace\n",
    "\n",
    "# Load the workspace from the saved config file\n",
    "ws = Workspace.from_config()\n",
    "print('Ready to use Azure ML {} to work with {}'.format(azureml.core.VERSION, ws.name))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create a Training Script\n",
    "\n",
    "You're going to use a Python script to train a machine learning model based on the diabates data, so let's start by creating a folder for the script and data files."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'diabetes-training\\\\diabetes.csv'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import os, shutil\n",
    "\n",
    "# Create a folder for the experiment files\n",
    "training_folder = 'diabetes-training'\n",
    "os.makedirs(training_folder, exist_ok=True)\n",
    "\n",
    "# Copy the data file into the experiment folder\n",
    "shutil.copy('data/diabetes.csv', os.path.join(training_folder, \"diabetes.csv\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now you're ready to create the training script and save it in the folder."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Writing diabetes-training/diabetes_training.py\n"
     ]
    }
   ],
   "source": [
    "%%writefile $training_folder/diabetes_training.py\n",
    "# Import libraries\n",
    "from azureml.core import Run\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import joblib\n",
    "import os\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.metrics import roc_auc_score\n",
    "from sklearn.metrics import roc_curve\n",
    "\n",
    "# Get the experiment run context\n",
    "run = Run.get_context()\n",
    "\n",
    "# load the diabetes dataset\n",
    "print(\"Loading Data...\")\n",
    "diabetes = pd.read_csv('diabetes.csv')\n",
    "\n",
    "# Separate features and labels\n",
    "X, y = diabetes[['Pregnancies','PlasmaGlucose','DiastolicBloodPressure','TricepsThickness','SerumInsulin','BMI','DiabetesPedigree','Age']].values, diabetes['Diabetic'].values\n",
    "\n",
    "# Split data into training set and test set\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.30, random_state=0)\n",
    "\n",
    "# Set regularization hyperparameter\n",
    "reg = 0.01\n",
    "\n",
    "# Train a logistic regression model\n",
    "print('Training a logistic regression model with regularization rate of', reg)\n",
    "run.log('Regularization Rate',  np.float(reg))\n",
    "model = LogisticRegression(C=1/reg, solver=\"liblinear\").fit(X_train, y_train)\n",
    "\n",
    "# calculate accuracy\n",
    "y_hat = model.predict(X_test)\n",
    "acc = np.average(y_hat == y_test)\n",
    "print('Accuracy:', acc)\n",
    "run.log('Accuracy', np.float(acc))\n",
    "\n",
    "# calculate AUC\n",
    "y_scores = model.predict_proba(X_test)\n",
    "auc = roc_auc_score(y_test,y_scores[:,1])\n",
    "print('AUC: ' + str(auc))\n",
    "run.log('AUC', np.float(auc))\n",
    "\n",
    "# Save the trained model in the outputs folder\n",
    "os.makedirs('outputs', exist_ok=True)\n",
    "joblib.dump(value=model, filename='outputs/diabetes_model.pkl')\n",
    "\n",
    "run.complete()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Use an Estimator to Run the Script as an Experiment\n",
    "\n",
    "You can run experiment scripts using a **RunConfiguration** and a **ScriptRunConfig**, or you can use an **Estimator**, which abstracts both of these configurations in a single object.\n",
    "\n",
    "In this case, we'll use a generic **Estimator** object to run the training experiment. Note that the default environment for this estimator does not include the **scikit-learn** package, so you need to explicitly add that to the configuration. The conda environment is built on-demand the first time the estimator is used, and cached for future runs that use the same configuration; so the first run will take a little longer. On subsequent runs, the cached environment can be re-used so they'll complete more quickly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "2a7d816de3be4aa39360ae58903617dd",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "_UserRunWidget(widget_settings={'childWidgetDisplay': 'popup', 'send_telemetry': False, 'log_level': 'INFO', '…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/aml.mini.widget.v1": "{\"status\": \"Completed\", \"workbench_run_details_uri\": \"https://ml.azure.com/runs/diabetes-training_1649521014_99425c33?wsid=/subscriptions/8ef9d38c-cd92-4cea-b46c-0527c93d4acb/resourcegroups/azure-ml-sdk-project/workspaces/azureml-workspace&tid=babcc559-b87a-427b-9175-f021a70345c0\", \"run_id\": \"diabetes-training_1649521014_99425c33\", \"run_properties\": {\"run_id\": \"diabetes-training_1649521014_99425c33\", \"created_utc\": \"2022-04-09T16:16:54.195227Z\", \"properties\": {\"_azureml.ComputeTargetType\": \"local\", \"ContentSnapshotId\": \"fb6b5262-04cf-4199-8ef4-f2dbe6ca5100\", \"azureml.git.repository_uri\": \"https://github.com/ovokpus/Azure-Machine-Learning-Project.git\", \"mlflow.source.git.repoURL\": \"https://github.com/ovokpus/Azure-Machine-Learning-Project.git\", \"azureml.git.branch\": \"main\", \"mlflow.source.git.branch\": \"main\", \"azureml.git.commit\": \"d1aabb6aca40b9f8c61269b61b807d44e2309e31\", \"mlflow.source.git.commit\": \"d1aabb6aca40b9f8c61269b61b807d44e2309e31\", \"azureml.git.dirty\": \"False\"}, \"tags\": {}, \"script_name\": null, \"arguments\": null, \"end_time_utc\": \"2022-04-09T16:17:14.260357Z\", \"status\": \"Completed\", \"log_files\": {\"azureml-logs/60_control_log.txt\": \"https://azuremlworkspa4632104135.blob.core.windows.net/azureml/ExperimentRun/dcid.diabetes-training_1649521014_99425c33/azureml-logs/60_control_log.txt?sv=2019-07-07&sr=b&sig=49R14eXk1V91J3obRActEWPTU1BsvIiNNE6ECKJg09g%3D&skoid=f5abfce9-8646-42e6-99ea-e8297745b595&sktid=babcc559-b87a-427b-9175-f021a70345c0&skt=2022-04-09T09%3A49%3A47Z&ske=2022-04-10T17%3A59%3A47Z&sks=b&skv=2019-07-07&st=2022-04-09T16%3A17%3A31Z&se=2022-04-10T00%3A27%3A31Z&sp=r\", \"azureml-logs/70_driver_log.txt\": \"https://azuremlworkspa4632104135.blob.core.windows.net/azureml/ExperimentRun/dcid.diabetes-training_1649521014_99425c33/azureml-logs/70_driver_log.txt?sv=2019-07-07&sr=b&sig=X3Rz935gqFLnpi%2BylafuoTynVrIEEDcUWTD7wWpo85A%3D&skoid=f5abfce9-8646-42e6-99ea-e8297745b595&sktid=babcc559-b87a-427b-9175-f021a70345c0&skt=2022-04-09T09%3A49%3A47Z&ske=2022-04-10T17%3A59%3A47Z&sks=b&skv=2019-07-07&st=2022-04-09T16%3A17%3A31Z&se=2022-04-10T00%3A27%3A31Z&sp=r\", \"logs/azureml/9620_azureml.log\": \"https://azuremlworkspa4632104135.blob.core.windows.net/azureml/ExperimentRun/dcid.diabetes-training_1649521014_99425c33/logs/azureml/9620_azureml.log?sv=2019-07-07&sr=b&sig=mURnLQ%2Bf%2BuTcWWTCa9V3vAhGZbCjt97DIXhQ%2F6gL104%3D&skoid=f5abfce9-8646-42e6-99ea-e8297745b595&sktid=babcc559-b87a-427b-9175-f021a70345c0&skt=2022-04-09T09%3A49%3A47Z&ske=2022-04-10T17%3A59%3A47Z&sks=b&skv=2019-07-07&st=2022-04-09T16%3A17%3A11Z&se=2022-04-10T00%3A27%3A11Z&sp=r\"}, \"log_groups\": [[\"azureml-logs/60_control_log.txt\"], [\"azureml-logs/70_driver_log.txt\"], [\"logs/azureml/9620_azureml.log\"]], \"run_duration\": \"0:00:20\", \"run_number\": \"1649521014\", \"run_queued_details\": {\"status\": \"Completed\", \"details\": null}}, \"child_runs\": [], \"children_metrics\": {}, \"run_metrics\": [{\"name\": \"Regularization Rate\", \"run_id\": \"diabetes-training_1649521014_99425c33\", \"categories\": [0], \"series\": [{\"data\": [0.01]}]}, {\"name\": \"Accuracy\", \"run_id\": \"diabetes-training_1649521014_99425c33\", \"categories\": [0], \"series\": [{\"data\": [0.774]}]}, {\"name\": \"AUC\", \"run_id\": \"diabetes-training_1649521014_99425c33\", \"categories\": [0], \"series\": [{\"data\": [0.848370565699786]}]}], \"run_logs\": \"2022-04-09 10:16:59,444|azureml|DEBUG|Inputs:: kwargs: {'OutputCollection': True, 'EnableMLflowTracking': True, 'snapshotProject': True}, track_folders: None, deny_list: None, directories_to_watch: ['logs', 'logs/azureml']\\r\\n2022-04-09 10:16:59,446|azureml.history._tracking.PythonWorkingDirectory|DEBUG|Execution target type: none\\r\\n2022-04-09 10:16:59,448|azureml.history._tracking.PythonWorkingDirectory|DEBUG|Failed to import pyspark with error: No module named 'pyspark'\\r\\n2022-04-09 10:16:59,448|azureml.history._tracking.PythonWorkingDirectory.workingdir|DEBUG|Pinning working directory for filesystems: ['pyfs']\\r\\n2022-04-09 10:17:00,227|azureml.core._experiment_method|DEBUG|Trying to register submit_function search, on method <class 'azureml.train.hyperdrive.runconfig.HyperDriveRunConfig'>\\r\\n2022-04-09 10:17:00,227|azureml.core._experiment_method|DEBUG|Registered submit_function search, on method <class 'azureml.train.hyperdrive.runconfig.HyperDriveRunConfig'>\\r\\n2022-04-09 10:17:00,227|azureml.core._experiment_method|DEBUG|Trying to register submit_function search, on method <class 'azureml.train.hyperdrive.runconfig.HyperDriveConfig'>\\r\\n2022-04-09 10:17:00,227|azureml.core._experiment_method|DEBUG|Registered submit_function search, on method <class 'azureml.train.hyperdrive.runconfig.HyperDriveConfig'>\\r\\n2022-04-09 10:17:00,228|azureml.core.run|DEBUG|Adding new factory <function HyperDriveRun._from_run_dto at 0x0000027B6719B598> for run source hyperdrive\\r\\n2022-04-09 10:17:00,246|azureml.core.run|DEBUG|Adding new factory <function ScriptRun._from_run_dto at 0x0000027B66FC3620> for run source azureml.scriptrun\\r\\n2022-04-09 10:17:00,248|azureml.core.run|DEBUG|Identity in use is not set. Falling back to using AMLToken\\r\\n2022-04-09 10:17:00,248|azureml.core.run|DEBUG|Using AMLToken auth for remote run\\r\\n2022-04-09 10:17:00,255|azureml.core.authentication.TokenRefresherDaemon|DEBUG|Starting daemon and triggering first instance\\r\\n2022-04-09 10:17:00,261|azureml._restclient.clientbase|INFO|Created a worker pool for first use\\r\\n2022-04-09 10:17:00,261|azureml.core.authentication|DEBUG|Time to expire 1792793.738689 seconds\\r\\n2022-04-09 10:17:00,261|azureml._restclient.service_context|DEBUG|Created a static thread pool for ServiceContext class\\r\\n2022-04-09 10:17:00,262|azureml._base_sdk_common.service_discovery|DEBUG|Found history service url in environment variable AZUREML_SERVICE_ENDPOINT, history service url: https://westus2.api.azureml.ms.\\r\\n2022-04-09 10:17:00,263|azureml._base_sdk_common.service_discovery|DEBUG|Found history service url in environment variable AZUREML_SERVICE_ENDPOINT, history service url: https://westus2.api.azureml.ms.\\r\\n2022-04-09 10:17:00,264|azureml._base_sdk_common.service_discovery|DEBUG|Found history service url in environment variable AZUREML_SERVICE_ENDPOINT, history service url: https://westus2.api.azureml.ms.\\r\\n2022-04-09 10:17:00,264|azureml._base_sdk_common.service_discovery|DEBUG|Found history service url in environment variable AZUREML_SERVICE_ENDPOINT, history service url: https://westus2.api.azureml.ms.\\r\\n2022-04-09 10:17:00,265|azureml._base_sdk_common.service_discovery|DEBUG|Found history service url in environment variable AZUREML_SERVICE_ENDPOINT, history service url: https://westus2.api.azureml.ms.\\r\\n2022-04-09 10:17:00,265|azureml._base_sdk_common.service_discovery|DEBUG|Found history service url in environment variable AZUREML_SERVICE_ENDPOINT, history service url: https://westus2.api.azureml.ms.\\r\\n2022-04-09 10:17:00,265|azureml._base_sdk_common.service_discovery|DEBUG|Found history service url in environment variable AZUREML_SERVICE_ENDPOINT, history service url: https://westus2.api.azureml.ms.\\r\\n2022-04-09 10:17:00,420|azureml.core._metrics|DEBUG|numpy.float128 is unsupported, expected for windows\\r\\n2022-04-09 10:17:00,444|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.RunClient.get_by_exp_id-async:False|DEBUG|[START]\\r\\n2022-04-09 10:17:00,444|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.RunClient|DEBUG|ClientBase: Calling get_by_exp_id with url /history/v1.0/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.MachineLearningServices/workspaces/{workspaceName}/experimentids/{experimentId}/runs/{runId}\\r\\n2022-04-09 10:17:01,409|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.RunClient.get_by_exp_id-async:False|DEBUG|[STOP]\\r\\n2022-04-09 10:17:01,410|azureml._SubmittedRun#diabetes-training_1649521014_99425c33|DEBUG|Constructing run from dto. type: azureml.scriptrun, source: None, props: {'_azureml.ComputeTargetType': 'local', 'ContentSnapshotId': 'fb6b5262-04cf-4199-8ef4-f2dbe6ca5100', 'azureml.git.repository_uri': 'https://github.com/ovokpus/Azure-Machine-Learning-Project.git', 'mlflow.source.git.repoURL': 'https://github.com/ovokpus/Azure-Machine-Learning-Project.git', 'azureml.git.branch': 'main', 'mlflow.source.git.branch': 'main', 'azureml.git.commit': 'd1aabb6aca40b9f8c61269b61b807d44e2309e31', 'mlflow.source.git.commit': 'd1aabb6aca40b9f8c61269b61b807d44e2309e31', 'azureml.git.dirty': 'False'}\\r\\n2022-04-09 10:17:01,410|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunContextManager|DEBUG|Valid logs dir, setting up content loader\\r\\n2022-04-09 10:17:01,411|azureml|WARNING|Could not import azureml.mlflow or azureml.contrib.mlflow mlflow APIs will not run against AzureML services.  Add azureml-mlflow as a conda dependency for the run if this behavior is desired\\r\\n2022-04-09 10:17:01,412|azureml.WorkerPool|DEBUG|[START]\\r\\n2022-04-09 10:17:01,412|azureml.SendRunKillSignal|DEBUG|[START]\\r\\n2022-04-09 10:17:01,412|azureml.RunStatusContext|DEBUG|[START]\\r\\n2022-04-09 10:17:01,412|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunContextManager.RunStatusContext|DEBUG|[START]\\r\\n2022-04-09 10:17:01,412|azureml.MetricsClient|DEBUG|[START]\\r\\n2022-04-09 10:17:01,412|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.MetricsClient|DEBUG|[START]\\r\\n2022-04-09 10:17:01,412|azureml.ContentUploader|DEBUG|[START]\\r\\n2022-04-09 10:17:01,413|azureml._history.utils.context_managers|DEBUG|starting file watcher\\r\\n2022-04-09 10:17:01,413|azureml._history.utils.context_managers.FileWatcher.UploadQueue|DEBUG|[Start]\\r\\n2022-04-09 10:17:01,413|azureml.TrackFolders|DEBUG|[START]\\r\\n2022-04-09 10:17:01,413|azureml.WorkingDirectoryCM|DEBUG|[START]\\r\\n2022-04-09 10:17:01,414|azureml.history._tracking.PythonWorkingDirectory.workingdir|DEBUG|[START]\\r\\n2022-04-09 10:17:01,414|azureml.history._tracking.PythonWorkingDirectory|INFO|Current working dir: C:\\\\Users\\\\OVOOKP~1\\\\AppData\\\\Local\\\\Temp\\\\azureml_runs\\\\diabetes-training_1649521014_99425c33\\r\\n2022-04-09 10:17:01,414|azureml.history._tracking.PythonWorkingDirectory.workingdir|DEBUG|Calling pyfs\\r\\n2022-04-09 10:17:01,414|azureml.history._tracking.PythonWorkingDirectory.workingdir|DEBUG|Storing working dir for pyfs as C:\\\\Users\\\\OVOOKP~1\\\\AppData\\\\Local\\\\Temp\\\\azureml_runs\\\\diabetes-training_1649521014_99425c33\\r\\n2022-04-09 10:17:01,427|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.ArtifactsClient.batch_create_empty_artifacts-async:False|DEBUG|[START]\\r\\n2022-04-09 10:17:01,427|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.ArtifactsClient|DEBUG|ClientBase: Calling batch_create_empty_artifacts with url /artifact/v2.0/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.MachineLearningServices/workspaces/{workspaceName}/artifacts/batch/metadata/{origin}/{container}\\r\\n2022-04-09 10:17:01,865|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.ArtifactsClient.batch_create_empty_artifacts-async:False|DEBUG|[STOP]\\r\\n2022-04-09 10:17:02,196|azureml._history.utils.context_managers.FileWatcher|DEBUG|uploading data to container: azureml blob: ExperimentRun/dcid.diabetes-training_1649521014_99425c33/logs/azureml/9620_azureml.log path: C:\\\\Users\\\\OVOOKP~1\\\\AppData\\\\Local\\\\Temp\\\\azureml_runs\\\\diabetes-training_1649521014_99425c33\\\\logs\\\\azureml\\\\9620_azureml.log\\r\\n2022-04-09 10:17:02,196|azureml._history.utils.context_managers.FileWatcher.UploadQueue.WorkerPool|DEBUG|submitting future: result\\r\\n2022-04-09 10:17:02,469|azureml._history.utils.context_managers.FileWatcher.UploadQueue.0_result|DEBUG|Using basic handler - no exception handling\\r\\n2022-04-09 10:17:02,469|azureml._history.utils.context_managers.FileWatcher.UploadQueue|DEBUG|Adding task 0_result to queue of approximate size: 0\\r\\n2022-04-09 10:17:02,817|azureml.core.run|DEBUG|Identity in use is not set. Falling back to using AMLToken\\r\\n2022-04-09 10:17:02,817|azureml.core.run|DEBUG|Using AMLToken auth for remote run\\r\\n2022-04-09 10:17:02,818|azureml._restclient.service_context|DEBUG|Access an existing static threadpool for ServiceContext class\\r\\n2022-04-09 10:17:02,818|azureml._base_sdk_common.service_discovery|DEBUG|Found history service url in environment variable AZUREML_SERVICE_ENDPOINT, history service url: https://westus2.api.azureml.ms.\\r\\n2022-04-09 10:17:02,819|azureml._base_sdk_common.service_discovery|DEBUG|Found history service url in environment variable AZUREML_SERVICE_ENDPOINT, history service url: https://westus2.api.azureml.ms.\\r\\n2022-04-09 10:17:02,820|azureml._base_sdk_common.service_discovery|DEBUG|Found history service url in environment variable AZUREML_SERVICE_ENDPOINT, history service url: https://westus2.api.azureml.ms.\\r\\n2022-04-09 10:17:02,821|azureml._base_sdk_common.service_discovery|DEBUG|Found history service url in environment variable AZUREML_SERVICE_ENDPOINT, history service url: https://westus2.api.azureml.ms.\\r\\n2022-04-09 10:17:02,822|azureml._base_sdk_common.service_discovery|DEBUG|Found history service url in environment variable AZUREML_SERVICE_ENDPOINT, history service url: https://westus2.api.azureml.ms.\\r\\n2022-04-09 10:17:02,822|azureml._base_sdk_common.service_discovery|DEBUG|Found history service url in environment variable AZUREML_SERVICE_ENDPOINT, history service url: https://westus2.api.azureml.ms.\\r\\n2022-04-09 10:17:02,822|azureml._base_sdk_common.service_discovery|DEBUG|Found history service url in environment variable AZUREML_SERVICE_ENDPOINT, history service url: https://westus2.api.azureml.ms.\\r\\n2022-04-09 10:17:02,850|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.RunClient.get_by_exp_id-async:False|DEBUG|[START]\\r\\n2022-04-09 10:17:02,850|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.RunClient|DEBUG|ClientBase: Calling get_by_exp_id with url /history/v1.0/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.MachineLearningServices/workspaces/{workspaceName}/experimentids/{experimentId}/runs/{runId}\\r\\n2022-04-09 10:17:03,053|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.RunClient.get_by_exp_id-async:False|DEBUG|[STOP]\\r\\n2022-04-09 10:17:03,054|azureml._SubmittedRun#diabetes-training_1649521014_99425c33|DEBUG|Constructing run from dto. type: azureml.scriptrun, source: None, props: {'_azureml.ComputeTargetType': 'local', 'ContentSnapshotId': 'fb6b5262-04cf-4199-8ef4-f2dbe6ca5100', 'azureml.git.repository_uri': 'https://github.com/ovokpus/Azure-Machine-Learning-Project.git', 'mlflow.source.git.repoURL': 'https://github.com/ovokpus/Azure-Machine-Learning-Project.git', 'azureml.git.branch': 'main', 'mlflow.source.git.branch': 'main', 'azureml.git.commit': 'd1aabb6aca40b9f8c61269b61b807d44e2309e31', 'mlflow.source.git.commit': 'd1aabb6aca40b9f8c61269b61b807d44e2309e31', 'azureml.git.dirty': 'False'}\\r\\n2022-04-09 10:17:03,054|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunContextManager|DEBUG|Valid logs dir, setting up content loader\\r\\n2022-04-09 10:17:03,074|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.MetricsClient|DEBUG|Overrides: Max batch size: 50, batch cushion: 5, Interval: 1.\\r\\n2022-04-09 10:17:03,075|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.MetricsClient.PostMetricsBatchV2.PostMetricsBatchV2Daemon|DEBUG|Starting daemon and triggering first instance\\r\\n2022-04-09 10:17:03,084|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.MetricsClient|DEBUG|Used <class 'azureml._common.async_utils.batch_task_queue.BatchTaskQueue'> for use_batch=True.\\r\\n2022-04-09 10:17:03,126|azureml._SubmittedRun#diabetes-training_1649521014_99425c33|INFO|complete is not setting status for submitted runs.\\r\\n2022-04-09 10:17:03,126|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.MetricsClient.FlushingMetricsClient|DEBUG|[START]\\r\\n2022-04-09 10:17:03,126|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.MetricsClient|DEBUG|Overrides: Max batch size: 50, batch cushion: 5, Interval: 1.\\r\\n2022-04-09 10:17:03,126|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.MetricsClient.PostMetricsBatch.PostMetricsBatchDaemon|DEBUG|Starting daemon and triggering first instance\\r\\n2022-04-09 10:17:03,127|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.MetricsClient|DEBUG|Used <class 'azureml._common.async_utils.batch_task_queue.BatchTaskQueue'> for use_batch=True.\\r\\n2022-04-09 10:17:03,127|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.MetricsClient.PostMetricsBatch.WaitFlushSource:MetricsClient|DEBUG|[START]\\r\\n2022-04-09 10:17:03,127|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.MetricsClient.PostMetricsBatch.WaitFlushSource:MetricsClient|DEBUG|flush timeout 300 is different from task queue timeout 120, using flush timeout\\r\\n2022-04-09 10:17:03,127|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.MetricsClient.PostMetricsBatch.WaitFlushSource:MetricsClient|DEBUG|Waiting 300 seconds on tasks: [].\\r\\n2022-04-09 10:17:03,127|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.MetricsClient.PostMetricsBatch|DEBUG|\\r\\n2022-04-09 10:17:03,127|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.MetricsClient.PostMetricsBatch.WaitFlushSource:MetricsClient|DEBUG|[STOP]\\r\\n2022-04-09 10:17:03,127|azureml.BatchTaskQueueAdd_1_Batches|DEBUG|[Start]\\r\\n2022-04-09 10:17:03,127|azureml.BatchTaskQueueAdd_1_Batches.WorkerPool|DEBUG|submitting future: _handle_batch\\r\\n2022-04-09 10:17:03,128|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.MetricsClient.PostMetricsBatchV2|DEBUG|Batch size 3.\\r\\n2022-04-09 10:17:03,128|azureml.BatchTaskQueueAdd_1_Batches.0__handle_batch|DEBUG|Using basic handler - no exception handling\\r\\n2022-04-09 10:17:03,128|azureml._restclient.service_context.WorkerPool|DEBUG|submitting future: _log_batch_v2\\r\\n2022-04-09 10:17:03,128|azureml.BatchTaskQueueAdd_1_Batches|DEBUG|Adding task 0__handle_batch to queue of approximate size: 0\\r\\n2022-04-09 10:17:03,129|azureml.BatchTaskQueueAdd_1_Batches|DEBUG|[Stop] - waiting default timeout\\r\\n2022-04-09 10:17:03,129|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.MetricsClient|DEBUG|Metrics Client: _log_batch_v2 is calling post_run_metrics posting 3 values.\\r\\n2022-04-09 10:17:03,129|azureml.BatchTaskQueueAdd_1_Batches.WaitFlushSource:BatchTaskQueueAdd_1_Batches|DEBUG|[START]\\r\\n2022-04-09 10:17:03,129|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.MetricsClient.PostMetricsBatchV2.0__log_batch_v2|DEBUG|Using basic handler - no exception handling\\r\\n2022-04-09 10:17:03,129|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.MetricsClient._post_run_metrics_log_failed_validations-async:False|DEBUG|[START]\\r\\n2022-04-09 10:17:03,129|azureml.BatchTaskQueueAdd_1_Batches.WaitFlushSource:BatchTaskQueueAdd_1_Batches|DEBUG|Overriding default flush timeout from None to 120\\r\\n2022-04-09 10:17:03,129|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.MetricsClient.PostMetricsBatchV2|DEBUG|Adding task 0__log_batch_v2 to queue of approximate size: 0\\r\\n2022-04-09 10:17:03,129|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.MetricsClient|DEBUG|ClientBase: Calling _post_run_metrics_log_failed_validations with url None\\r\\n2022-04-09 10:17:03,129|azureml.BatchTaskQueueAdd_1_Batches.WaitFlushSource:BatchTaskQueueAdd_1_Batches|DEBUG|Waiting 120 seconds on tasks: [AsyncTask(0__handle_batch)].\\r\\n2022-04-09 10:17:03,131|azureml.BatchTaskQueueAdd_1_Batches.0__handle_batch.WaitingTask|DEBUG|[START]\\r\\n2022-04-09 10:17:03,131|azureml.BatchTaskQueueAdd_1_Batches.0__handle_batch.WaitingTask|DEBUG|Awaiter is BatchTaskQueueAdd_1_Batches\\r\\n2022-04-09 10:17:03,131|azureml.BatchTaskQueueAdd_1_Batches.0__handle_batch.WaitingTask|DEBUG|[STOP]\\r\\n2022-04-09 10:17:03,131|azureml.BatchTaskQueueAdd_1_Batches|DEBUG|\\r\\n2022-04-09 10:17:03,132|azureml.BatchTaskQueueAdd_1_Batches.WaitFlushSource:BatchTaskQueueAdd_1_Batches|DEBUG|[STOP]\\r\\n2022-04-09 10:17:03,132|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.MetricsClient.PostMetricsBatchV2.WaitFlushSource:MetricsClient|DEBUG|[START]\\r\\n2022-04-09 10:17:03,132|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.MetricsClient.PostMetricsBatchV2.WaitFlushSource:MetricsClient|DEBUG|flush timeout 300 is different from task queue timeout 120, using flush timeout\\r\\n2022-04-09 10:17:03,132|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.MetricsClient.PostMetricsBatchV2.WaitFlushSource:MetricsClient|DEBUG|Waiting 300 seconds on tasks: [AsyncTask(0__log_batch_v2)].\\r\\n2022-04-09 10:17:03,499|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.MetricsClient._post_run_metrics_log_failed_validations-async:False|DEBUG|[STOP]\\r\\n2022-04-09 10:17:03,648|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.MetricsClient.PostMetricsBatchV2.0__log_batch_v2.WaitingTask|DEBUG|[START]\\r\\n2022-04-09 10:17:03,648|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.MetricsClient.PostMetricsBatchV2.0__log_batch_v2.WaitingTask|DEBUG|Awaiter is PostMetricsBatchV2\\r\\n2022-04-09 10:17:03,648|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.MetricsClient.PostMetricsBatchV2.0__log_batch_v2.WaitingTask|DEBUG|[STOP]\\r\\n2022-04-09 10:17:03,648|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.MetricsClient.PostMetricsBatchV2|DEBUG|Waiting on task: 0__log_batch_v2.\\r\\n1 tasks left. Current duration of flush 0.0 seconds.\\r\\nWaiting on task: 0__log_batch_v2.\\r\\n1 tasks left. Current duration of flush 0.2508659362792969 seconds.\\r\\n\\r\\n2022-04-09 10:17:03,648|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.MetricsClient.PostMetricsBatchV2.WaitFlushSource:MetricsClient|DEBUG|[STOP]\\r\\n2022-04-09 10:17:03,648|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.MetricsClient.FlushingMetricsClient|DEBUG|[STOP]\\r\\n2022-04-09 10:17:03,648|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.MetricsClient.wait_on_ingest-async:False|DEBUG|[START]\\r\\n2022-04-09 10:17:03,648|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.MetricsClient|DEBUG|ClientBase: Calling wait_on_ingest with url /history/v1.0/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.MachineLearningServices/workspaces/{workspaceName}/experiments/{experimentName}/runs/{runId}/metricsingest/wait\\r\\n2022-04-09 10:17:03,869|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.MetricsClient.wait_on_ingest-async:False|DEBUG|[STOP]\\r\\n2022-04-09 10:17:09,148|azureml._restclient.clientbase|DEBUG|ClientBase: Calling update_status with url None\\r\\n2022-04-09 10:17:09,436|azureml.history._tracking.PythonWorkingDirectory.workingdir|DEBUG|Uploading tracked directories: [], excluding []\\r\\n2022-04-09 10:17:09,437|azureml.history._tracking.PythonWorkingDirectory.workingdir|DEBUG|Calling track for pyfs\\r\\n2022-04-09 10:17:09,706|azureml.history._tracking.PythonWorkingDirectory.workingdir|DEBUG|Calling pyfs\\r\\n2022-04-09 10:17:09,706|azureml.history._tracking.PythonWorkingDirectory|INFO|Current working dir: C:\\\\Users\\\\OVOOKP~1\\\\AppData\\\\Local\\\\Temp\\\\azureml_runs\\\\diabetes-training_1649521014_99425c33\\r\\n2022-04-09 10:17:09,706|azureml.history._tracking.PythonWorkingDirectory.workingdir|DEBUG|Reverting working dir from C:\\\\Users\\\\OVOOKP~1\\\\AppData\\\\Local\\\\Temp\\\\azureml_runs\\\\diabetes-training_1649521014_99425c33 to C:\\\\Users\\\\OVOOKP~1\\\\AppData\\\\Local\\\\Temp\\\\azureml_runs\\\\diabetes-training_1649521014_99425c33\\r\\n2022-04-09 10:17:09,706|azureml.history._tracking.PythonWorkingDirectory|INFO|Working dir is already updated C:\\\\Users\\\\OVOOKP~1\\\\AppData\\\\Local\\\\Temp\\\\azureml_runs\\\\diabetes-training_1649521014_99425c33\\r\\n2022-04-09 10:17:09,706|azureml.history._tracking.PythonWorkingDirectory.workingdir|DEBUG|[STOP]\\r\\n2022-04-09 10:17:09,706|azureml.WorkingDirectoryCM|DEBUG|[STOP]\\r\\n2022-04-09 10:17:09,706|azureml.history._tracking.PythonWorkingDirectory.workingdir|DEBUG|Uploading tracked directories: ['./outputs'], excluding ['azureml-logs/driver_log']\\r\\n2022-04-09 10:17:09,706|azureml.history._tracking.PythonWorkingDirectory.workingdir|DEBUG|Calling track for pyfs\\r\\n2022-04-09 10:17:09,706|azureml.history._tracking.PythonWorkingDirectory|DEBUG|./outputs exists as directory, uploading..\\r\\n2022-04-09 10:17:09,706|azureml.history._tracking.PythonWorkingDirectory|DEBUG|Found and adding path to upload: ./outputs\\\\diabetes_model.pkl\\r\\n2022-04-09 10:17:09,707|azureml.history._tracking.PythonWorkingDirectory|DEBUG|Paths to upload is ['./outputs\\\\\\\\diabetes_model.pkl'] in dir ./outputs\\r\\n2022-04-09 10:17:09,707|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.ArtifactsClient.upload_files|DEBUG|Overriding default timeout to 300\\r\\n2022-04-09 10:17:09,707|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.ArtifactsClient.upload_files|DEBUG|[Start]\\r\\n2022-04-09 10:17:09,707|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.ArtifactsClient.batch_create_empty_artifacts-async:False|DEBUG|[START]\\r\\n2022-04-09 10:17:09,707|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.ArtifactsClient|DEBUG|ClientBase: Calling batch_create_empty_artifacts with url /artifact/v2.0/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.MachineLearningServices/workspaces/{workspaceName}/artifacts/batch/metadata/{origin}/{container}\\r\\n2022-04-09 10:17:10,017|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.ArtifactsClient.batch_create_empty_artifacts-async:False|DEBUG|[STOP]\\r\\n2022-04-09 10:17:10,017|azureml._restclient.service_context.WorkerPool|DEBUG|submitting future: perform_upload\\r\\n2022-04-09 10:17:10,018|azureml._restclient.clientbase|DEBUG|ClientBase: Calling create_blob_from_stream with url None\\r\\n2022-04-09 10:17:10,018|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.ArtifactsClient.upload_files.0_perform_upload|DEBUG|Using basic handler - no exception handling\\r\\n2022-04-09 10:17:10,018|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.ArtifactsClient.upload_files|DEBUG|Adding task 0_perform_upload to queue of approximate size: 0\\r\\n2022-04-09 10:17:10,018|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.ArtifactsClient.upload_files|DEBUG|[Stop] - waiting default timeout\\r\\n2022-04-09 10:17:10,018|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.ArtifactsClient.upload_files.WaitFlushSource:upload_files|DEBUG|[START]\\r\\n2022-04-09 10:17:10,018|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.ArtifactsClient.upload_files.WaitFlushSource:upload_files|DEBUG|Overriding default flush timeout from None to 300\\r\\n2022-04-09 10:17:10,019|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.ArtifactsClient.upload_files.WaitFlushSource:upload_files|DEBUG|Waiting 300 seconds on tasks: [AsyncTask(0_perform_upload)].\\r\\n2022-04-09 10:17:10,378|azureml._file_utils.upload|DEBUG|Uploaded blob ExperimentRun/dcid.diabetes-training_1649521014_99425c33/outputs/diabetes_model.pkl with size 964, file size 964.\\r\\n2022-04-09 10:17:10,535|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.ArtifactsClient.upload_files.0_perform_upload.WaitingTask|DEBUG|[START]\\r\\n2022-04-09 10:17:10,535|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.ArtifactsClient.upload_files.0_perform_upload.WaitingTask|DEBUG|Awaiter is upload_files\\r\\n2022-04-09 10:17:10,535|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.ArtifactsClient.upload_files.0_perform_upload.WaitingTask|DEBUG|[STOP]\\r\\n2022-04-09 10:17:10,535|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.ArtifactsClient.upload_files|DEBUG|Waiting on task: 0_perform_upload.\\r\\n1 tasks left. Current duration of flush 0.0010008811950683594 seconds.\\r\\nWaiting on task: 0_perform_upload.\\r\\n1 tasks left. Current duration of flush 0.2665243148803711 seconds.\\r\\n\\r\\n2022-04-09 10:17:10,535|azureml._SubmittedRun#diabetes-training_1649521014_99425c33.RunHistoryFacade.ArtifactsClient.upload_files.WaitFlushSource:upload_files|DEBUG|[STOP]\\r\\n2022-04-09 10:17:10,535|azureml.TrackFolders|DEBUG|[STOP]\\r\\n2022-04-09 10:17:10,535|azureml._history.utils.context_managers|DEBUG|exiting ContentUploader, waiting for file_watcher to finish upload...\\r\\n2022-04-09 10:17:10,535|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher called finish, setting event\\r\\n2022-04-09 10:17:10,535|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher received exit event, getting current_stat\\r\\n2022-04-09 10:17:10,536|azureml._history.utils.context_managers.FileWatcher.UploadQueue.WorkerPool|DEBUG|submitting future: result\\r\\n2022-04-09 10:17:10,537|azureml._history.utils.context_managers.FileWatcher.UploadQueue.1_result|DEBUG|Using basic handler - no exception handling\\r\\n2022-04-09 10:17:10,537|azureml._history.utils.context_managers.FileWatcher.UploadQueue|DEBUG|Adding task 1_result to queue of approximate size: 1\\r\\n2022-04-09 10:17:10,538|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher retrieved current_stat, will upload to current_stat\\r\\n2022-04-09 10:17:10,538|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,542|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,544|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,544|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,545|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,546|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,547|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,548|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,548|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,549|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,549|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,550|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,551|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,551|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,552|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,552|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,553|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,554|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,554|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,555|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,555|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,556|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,556|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,557|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,557|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,558|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,558|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,559|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,559|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,560|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,560|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,561|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,561|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,562|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,564|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,564|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,565|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,565|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,566|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,567|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,567|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,568|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,569|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,569|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,570|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,571|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,571|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,572|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,572|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,573|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,573|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,574|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,574|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,575|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,575|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,576|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,576|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,577|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,578|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,578|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,579|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,580|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,581|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,582|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,582|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,583|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,584|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,584|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,585|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,585|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,586|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,587|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,587|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,588|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,588|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,589|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,589|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,590|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,590|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,591|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,591|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,592|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,592|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,593|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,593|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,594|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,594|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,595|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,596|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,597|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,598|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,599|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,599|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,600|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,600|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,601|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,602|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,602|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,603|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,603|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,604|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,604|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,605|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,605|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,606|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,606|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,607|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,607|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,608|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,608|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,609|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,610|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,610|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,611|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,611|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,612|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,613|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,614|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,614|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,615|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,615|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,616|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,617|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,617|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,618|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,618|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,619|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,619|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,620|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,620|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,621|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,621|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,622|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,622|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,623|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,623|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,624|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,624|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,625|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,625|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,626|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,626|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,627|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,627|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,628|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,629|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,630|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,631|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,631|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,632|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,632|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,633|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,634|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,634|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,635|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,635|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,636|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,636|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,637|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,637|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,638|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,638|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,639|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,639|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,640|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,640|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,641|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,641|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,642|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,642|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,643|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,643|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,644|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,645|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,646|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,647|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,648|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,648|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,649|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,649|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,650|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,650|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,651|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,652|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,652|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,653|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,653|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,654|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,654|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,655|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,655|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,656|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,656|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,657|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,658|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,658|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,659|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,659|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,660|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,660|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,661|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,661|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,663|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,664|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,664|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,665|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,665|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,666|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,667|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,667|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,668|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,668|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,669|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,669|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,670|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,670|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,671|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,671|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,672|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,673|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,673|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,674|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,674|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,675|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,675|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,676|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,677|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,677|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,678|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,678|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,680|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,680|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,681|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,682|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,682|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,683|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,683|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,684|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,684|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,685|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,685|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,686|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,686|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,687|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,688|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,688|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,689|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,689|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,690|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,690|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,691|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,691|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,692|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,692|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,693|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,693|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,694|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,694|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,695|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,696|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,697|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,697|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,698|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,699|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,699|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,700|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,700|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,701|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,702|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,702|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,703|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,703|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,704|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,704|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,712|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,713|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,714|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,714|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,715|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,715|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,716|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,717|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,717|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,718|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,718|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,719|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,719|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,720|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,721|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,721|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,722|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,722|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,723|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,723|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,724|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,724|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,725|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,725|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,726|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,726|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,727|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,727|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,728|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,729|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,730|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,730|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,731|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,732|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,732|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,733|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,733|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,734|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,734|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,735|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,736|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,736|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,737|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,737|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,738|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,738|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,739|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,739|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,740|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,740|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,741|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,742|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,742|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,743|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,743|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,744|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,744|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,745|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,746|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,746|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,747|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,748|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,748|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,749|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,750|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,750|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,751|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,752|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,752|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,753|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,753|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,754|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:17:10,756|azureml._history.utils.context_managers.FileWatcher.UploadQueue.WorkerPool|DEBUG|submitting future: result\\r\\n2022-04-09 10:17:10,756|azureml._history.utils.context_managers.FileWatcher.UploadQueue.2_result|DEBUG|Using basic handler - no exception handling\\r\\n2022-04-09 10:17:10,756|azureml._history.utils.context_managers.FileWatcher.UploadQueue|DEBUG|Adding task 2_result to queue of approximate size: 2\\r\\n2022-04-09 10:17:10,757|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher finished uploading to current_stat, finishing task queue\\r\\n2022-04-09 10:17:10,757|azureml._history.utils.context_managers.FileWatcher.UploadQueue|DEBUG|[Stop] - waiting default timeout\\r\\n2022-04-09 10:17:10,757|azureml._history.utils.context_managers.FileWatcher.UploadQueue.WaitFlushSource:UploadQueue|DEBUG|[START]\\r\\n2022-04-09 10:17:10,757|azureml._history.utils.context_managers.FileWatcher.UploadQueue.WaitFlushSource:UploadQueue|DEBUG|Overriding default flush timeout from None to 120\\r\\n2022-04-09 10:17:10,757|azureml._history.utils.context_managers.FileWatcher.UploadQueue.WaitFlushSource:UploadQueue|DEBUG|Waiting 120 seconds on tasks: [AsyncTask(0_result), AsyncTask(1_result), AsyncTask(2_result)].\\r\\n2022-04-09 10:17:10,757|azureml._history.utils.context_managers.FileWatcher.UploadQueue.0_result.WaitingTask|DEBUG|[START]\\r\\n2022-04-09 10:17:10,757|azureml._history.utils.context_managers.FileWatcher.UploadQueue.0_result.WaitingTask|DEBUG|Awaiter is UploadQueue\\r\\n2022-04-09 10:17:10,757|azureml._history.utils.context_managers.FileWatcher.UploadQueue.0_result.WaitingTask|DEBUG|[STOP]\\r\\n2022-04-09 10:17:10,757|azureml._history.utils.context_managers.FileWatcher.UploadQueue.1_result.WaitingTask|DEBUG|[START]\\r\\n2022-04-09 10:17:10,757|azureml._history.utils.context_managers.FileWatcher.UploadQueue.1_result.WaitingTask|DEBUG|Awaiter is UploadQueue\\r\\n2022-04-09 10:17:10,757|azureml._history.utils.context_managers.FileWatcher.UploadQueue.1_result.WaitingTask|DEBUG|[STOP]\\r\\n2022-04-09 10:17:11,014|azureml._history.utils.context_managers.FileWatcher.UploadQueue.2_result.WaitingTask|DEBUG|[START]\\r\\n2022-04-09 10:17:11,014|azureml._history.utils.context_managers.FileWatcher.UploadQueue.2_result.WaitingTask|DEBUG|Awaiter is UploadQueue\\r\\n2022-04-09 10:17:11,014|azureml._history.utils.context_managers.FileWatcher.UploadQueue.2_result.WaitingTask|DEBUG|[STOP]\\r\\n2022-04-09 10:17:11,014|azureml._history.utils.context_managers.FileWatcher.UploadQueue|DEBUG|Waiting on task: 2_result.\\r\\n1 tasks left. Current duration of flush 0.0 seconds.\\r\\n\\r\\n2022-04-09 10:17:11,014|azureml._history.utils.context_managers.FileWatcher.UploadQueue.WaitFlushSource:UploadQueue|DEBUG|[STOP]\\r\\n\\nRun is completed.\", \"graph\": {}, \"widget_settings\": {\"childWidgetDisplay\": \"popup\", \"send_telemetry\": false, \"log_level\": \"INFO\", \"sdk_version\": \"1.40.0\"}, \"loading\": false}"
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RunId: diabetes-training_1649521014_99425c33\n",
      "Web View: https://ml.azure.com/runs/diabetes-training_1649521014_99425c33?wsid=/subscriptions/8ef9d38c-cd92-4cea-b46c-0527c93d4acb/resourcegroups/azure-ml-sdk-project/workspaces/azureml-workspace&tid=babcc559-b87a-427b-9175-f021a70345c0\n",
      "\n",
      "Streaming azureml-logs/70_driver_log.txt\n",
      "========================================\n",
      "\n",
      "[2022-04-09T16:16:58.393308] Entering context manager injector.\n",
      "[2022-04-09T16:16:59.434697] context_manager_injector.py Command line Options: Namespace(inject=['ProjectPythonPath:context_managers.ProjectPythonPath', 'RunHistory:context_managers.RunHistory', 'TrackUserError:context_managers.TrackUserError'], invocation=['diabetes_training.py'])\n",
      "Script type = None\n",
      "[2022-04-09T16:16:59.441704] Entering Run History Context Manager.\n",
      "[2022-04-09T16:17:01.414806] Current directory: C:\\Users\\OVOOKP~1\\AppData\\Local\\Temp\\azureml_runs\\diabetes-training_1649521014_99425c33\n",
      "[2022-04-09T16:17:01.414806] Preparing to call script [diabetes_training.py] with arguments:[]\n",
      "[2022-04-09T16:17:01.414806] After variable expansion, calling script [diabetes_training.py] with arguments:[]\n",
      "\n",
      "Loading Data...\n",
      "Training a logistic regression model with regularization rate of 0.01\n",
      "Accuracy: 0.774\n",
      "AUC: 0.848370565699786\n",
      "\n",
      "\n",
      "[2022-04-09T16:17:09.438328] The experiment completed successfully. Finalizing run...\n",
      "[2022-04-09T16:17:09.438328] Start FinalizingInRunHistory\n",
      "[2022-04-09T16:17:09.447327] Logging experiment finalizing status in history service.\n",
      "Starting the daemon thread to refresh tokens in background for process with pid = 9620\n",
      "Cleaning up all outstanding Run operations, waiting 300.0 seconds\n",
      "2 items cleaning up...\n",
      "Cleanup took 0.49929094314575195 seconds\n",
      "[2022-04-09T16:17:12.025145] Finished context manager injector.\n",
      "\n",
      "Execution Summary\n",
      "=================\n",
      "RunId: diabetes-training_1649521014_99425c33\n",
      "Web View: https://ml.azure.com/runs/diabetes-training_1649521014_99425c33?wsid=/subscriptions/8ef9d38c-cd92-4cea-b46c-0527c93d4acb/resourcegroups/azure-ml-sdk-project/workspaces/azureml-workspace&tid=babcc559-b87a-427b-9175-f021a70345c0\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'runId': 'diabetes-training_1649521014_99425c33',\n",
       " 'target': 'local',\n",
       " 'status': 'Completed',\n",
       " 'startTimeUtc': '2022-04-09T16:16:57.449055Z',\n",
       " 'endTimeUtc': '2022-04-09T16:17:14.260357Z',\n",
       " 'services': {},\n",
       " 'properties': {'_azureml.ComputeTargetType': 'local',\n",
       "  'ContentSnapshotId': 'fb6b5262-04cf-4199-8ef4-f2dbe6ca5100',\n",
       "  'azureml.git.repository_uri': 'https://github.com/ovokpus/Azure-Machine-Learning-Project.git',\n",
       "  'mlflow.source.git.repoURL': 'https://github.com/ovokpus/Azure-Machine-Learning-Project.git',\n",
       "  'azureml.git.branch': 'main',\n",
       "  'mlflow.source.git.branch': 'main',\n",
       "  'azureml.git.commit': 'd1aabb6aca40b9f8c61269b61b807d44e2309e31',\n",
       "  'mlflow.source.git.commit': 'd1aabb6aca40b9f8c61269b61b807d44e2309e31',\n",
       "  'azureml.git.dirty': 'False'},\n",
       " 'inputDatasets': [],\n",
       " 'outputDatasets': [],\n",
       " 'runDefinition': {'script': 'diabetes_training.py',\n",
       "  'command': '',\n",
       "  'useAbsolutePath': False,\n",
       "  'arguments': [],\n",
       "  'sourceDirectoryDataStore': None,\n",
       "  'framework': 'Python',\n",
       "  'communicator': 'None',\n",
       "  'target': 'local',\n",
       "  'dataReferences': {},\n",
       "  'data': {},\n",
       "  'outputData': {},\n",
       "  'datacaches': [],\n",
       "  'jobName': None,\n",
       "  'maxRunDurationSeconds': 2592000,\n",
       "  'nodeCount': 1,\n",
       "  'instanceTypes': [],\n",
       "  'priority': None,\n",
       "  'credentialPassthrough': False,\n",
       "  'identity': None,\n",
       "  'environment': {'name': 'ovo-azureml',\n",
       "   'version': 'Autosave_2022-04-09T16:14:18Z_5aba36a1',\n",
       "   'python': {'interpreterPath': 'python',\n",
       "    'userManagedDependencies': True,\n",
       "    'condaDependencies': {'channels': ['anaconda', 'conda-forge'],\n",
       "     'dependencies': ['python=3.6.2',\n",
       "      {'pip': ['adal==1.2.7',\n",
       "        'applicationinsights==0.11.10',\n",
       "        'argcomplete==1.12.3',\n",
       "        'argon2-cffi==21.3.0',\n",
       "        'argon2-cffi-bindings==21.2.0',\n",
       "        'async-generator==1.10',\n",
       "        'attrs==21.4.0',\n",
       "        'azure-common==1.1.28',\n",
       "        'azure-core==1.21.1',\n",
       "        'azure-graphrbac==0.61.1',\n",
       "        'azure-mgmt-authorization==0.61.0',\n",
       "        'azure-mgmt-containerregistry==8.2.0',\n",
       "        'azure-mgmt-core==1.3.0',\n",
       "        'azure-mgmt-keyvault==9.3.0',\n",
       "        'azure-mgmt-resource==20.1.0',\n",
       "        'azure-mgmt-storage==19.1.0',\n",
       "        'azure-storage-blob==12.11.0',\n",
       "        'azureml-core==1.40.0',\n",
       "        'azureml-telemetry==1.40.0',\n",
       "        'azureml-train==1.40.0',\n",
       "        'azureml-train-core==1.40.0',\n",
       "        'azureml-train-restclients-hyperdrive==1.40.0',\n",
       "        'azureml-widgets==1.40.0',\n",
       "        'backcall==0.2.0',\n",
       "        'backports.tempfile==1.0',\n",
       "        'backports.weakref==1.0.post1',\n",
       "        'bcrypt==3.2.0',\n",
       "        'bleach==4.1.0',\n",
       "        'certifi==2020.6.20',\n",
       "        'cffi==1.15.0',\n",
       "        'charset-normalizer==2.0.12',\n",
       "        'colorama==0.4.4',\n",
       "        'contextlib2==21.6.0',\n",
       "        'cryptography==36.0.2',\n",
       "        'cycler==0.11.0',\n",
       "        'dataclasses==0.8',\n",
       "        'decorator==5.1.1',\n",
       "        'defusedxml==0.7.1',\n",
       "        'docker==5.0.3',\n",
       "        'entrypoints==0.4',\n",
       "        'fairness-indicators==0.1.2',\n",
       "        'humanfriendly==10.0',\n",
       "        'idna==3.3',\n",
       "        'importlib-metadata==4.8.3',\n",
       "        'ipykernel==5.5.6',\n",
       "        'ipython==7.16.3',\n",
       "        'ipython-genutils==0.2.0',\n",
       "        'ipywidgets==7.7.0',\n",
       "        'isodate==0.6.1',\n",
       "        'jedi==0.17.2',\n",
       "        'jeepney==0.7.1',\n",
       "        'Jinja2==3.0.3',\n",
       "        'jmespath==0.10.0',\n",
       "        'joblib==1.1.0',\n",
       "        'jsonpickle==2.1.0',\n",
       "        'jsonschema==3.2.0',\n",
       "        'jupyter-client==7.1.2',\n",
       "        'jupyter-core==4.9.2',\n",
       "        'jupyterlab-pygments==0.1.2',\n",
       "        'jupyterlab-widgets==1.1.0',\n",
       "        'kiwisolver==1.3.1',\n",
       "        'knack==0.9.0',\n",
       "        'MarkupSafe==2.0.1',\n",
       "        'matplotlib==3.3.4',\n",
       "        'mistune==0.8.4',\n",
       "        'msal==1.17.0',\n",
       "        'msal-extensions==0.3.1',\n",
       "        'msrest==0.6.21',\n",
       "        'msrestazure==0.6.4',\n",
       "        'nbclient==0.5.9',\n",
       "        'nbconvert==6.0.7',\n",
       "        'nbformat==5.1.3',\n",
       "        'ndg-httpsclient==0.5.1',\n",
       "        'nest-asyncio==1.5.5',\n",
       "        'notebook==6.4.10',\n",
       "        'numpy==1.19.5',\n",
       "        'oauthlib==3.2.0',\n",
       "        'packaging==21.3',\n",
       "        'pandas==1.1.5',\n",
       "        'pandocfilters==1.5.0',\n",
       "        'paramiko==2.10.3',\n",
       "        'parso==0.7.1',\n",
       "        'pathspec==0.9.0',\n",
       "        'pickleshare==0.7.5',\n",
       "        'Pillow==8.4.0',\n",
       "        'pkginfo==1.8.2',\n",
       "        'portalocker==2.4.0',\n",
       "        'prometheus-client==0.14.1',\n",
       "        'prompt-toolkit==3.0.29',\n",
       "        'pyasn1==0.4.8',\n",
       "        'pycparser==2.21',\n",
       "        'Pygments==2.11.2',\n",
       "        'PyJWT==2.3.0',\n",
       "        'PyNaCl==1.5.0',\n",
       "        'pyOpenSSL==21.0.0',\n",
       "        'pyparsing==3.0.7',\n",
       "        'pyreadline==2.1',\n",
       "        'pyrsistent==0.18.0',\n",
       "        'PySocks==1.7.1',\n",
       "        'python-dateutil==2.8.2',\n",
       "        'pytz==2022.1',\n",
       "        'pywin32==227',\n",
       "        'pywinpty==1.1.6',\n",
       "        'PyYAML==6.0',\n",
       "        'pyzmq==22.3.0',\n",
       "        'requests==2.27.1',\n",
       "        'requests-oauthlib==1.3.1',\n",
       "        'scikit-learn==0.24.2',\n",
       "        'scipy==1.5.4',\n",
       "        'SecretStorage==3.3.1',\n",
       "        'Send2Trash==1.8.0',\n",
       "        'six==1.16.0',\n",
       "        'sklearn==0.0',\n",
       "        'tabulate==0.8.9',\n",
       "        'tensorflow-metadata==0.30.0',\n",
       "        'terminado==0.12.1',\n",
       "        'testpath==0.6.0',\n",
       "        'threadpoolctl==3.1.0',\n",
       "        'tornado==6.1',\n",
       "        'traitlets==4.3.3',\n",
       "        'typing_extensions==4.1.1',\n",
       "        'urllib3==1.26.7',\n",
       "        'wcwidth==0.2.5',\n",
       "        'webencodings==0.5.1',\n",
       "        'websocket-client==1.3.1',\n",
       "        'widgetsnbextension==3.6.0',\n",
       "        'wincertstore==0.2',\n",
       "        'zipp==3.6.0']},\n",
       "      'pip'],\n",
       "     'name': 'project_environment'},\n",
       "    'baseCondaEnvironment': None},\n",
       "   'environmentVariables': {'EXAMPLE_ENV_VAR': 'EXAMPLE_VALUE'},\n",
       "   'docker': {'baseImage': 'mcr.microsoft.com/azureml/openmpi3.1.2-ubuntu18.04:20220314.v1',\n",
       "    'platform': {'os': 'Linux', 'architecture': 'amd64'},\n",
       "    'baseDockerfile': None,\n",
       "    'baseImageRegistry': {'address': None, 'username': None, 'password': None},\n",
       "    'enabled': False,\n",
       "    'arguments': []},\n",
       "   'spark': {'repositories': [], 'packages': [], 'precachePackages': True},\n",
       "   'inferencingStackVersion': None},\n",
       "  'history': {'outputCollection': True,\n",
       "   'directoriesToWatch': ['logs'],\n",
       "   'enableMLflowTracking': True,\n",
       "   'snapshotProject': True},\n",
       "  'spark': {'configuration': {'spark.app.name': 'Azure ML Experiment',\n",
       "    'spark.yarn.maxAppAttempts': '1'}},\n",
       "  'parallelTask': {'maxRetriesPerWorker': 0,\n",
       "   'workerCountPerNode': 1,\n",
       "   'terminalExitCodes': None,\n",
       "   'configuration': {}},\n",
       "  'amlCompute': {'name': None,\n",
       "   'vmSize': None,\n",
       "   'retainCluster': False,\n",
       "   'clusterMaxNodeCount': None},\n",
       "  'aiSuperComputer': {'instanceType': 'D2',\n",
       "   'imageVersion': 'pytorch-1.7.0',\n",
       "   'location': None,\n",
       "   'aiSuperComputerStorageData': None,\n",
       "   'interactive': False,\n",
       "   'scalePolicy': None,\n",
       "   'virtualClusterArmId': None,\n",
       "   'tensorboardLogDirectory': None,\n",
       "   'sshPublicKey': None,\n",
       "   'sshPublicKeys': None,\n",
       "   'enableAzmlInt': True,\n",
       "   'priority': 'Medium',\n",
       "   'slaTier': 'Standard',\n",
       "   'userAlias': None},\n",
       "  'kubernetesCompute': {'instanceType': None},\n",
       "  'tensorflow': {'workerCount': 1, 'parameterServerCount': 1},\n",
       "  'mpi': {'processCountPerNode': 1},\n",
       "  'pyTorch': {'communicationBackend': 'nccl', 'processCount': None},\n",
       "  'hdi': {'yarnDeployMode': 'Cluster'},\n",
       "  'containerInstance': {'region': None, 'cpuCores': 2.0, 'memoryGb': 3.5},\n",
       "  'exposedPorts': None,\n",
       "  'docker': {'useDocker': False,\n",
       "   'sharedVolumes': True,\n",
       "   'shmSize': '2g',\n",
       "   'arguments': []},\n",
       "  'cmk8sCompute': {'configuration': {}},\n",
       "  'commandReturnCodeConfig': {'returnCode': 'Zero',\n",
       "   'successfulReturnCodes': []},\n",
       "  'environmentVariables': {},\n",
       "  'applicationEndpoints': {},\n",
       "  'parameters': []},\n",
       " 'logFiles': {'azureml-logs/60_control_log.txt': 'https://azuremlworkspa4632104135.blob.core.windows.net/azureml/ExperimentRun/dcid.diabetes-training_1649521014_99425c33/azureml-logs/60_control_log.txt?sv=2019-07-07&sr=b&sig=EAvNB02sSjVRPe1PMpdTLb2u9Zpz7KA3eQGgaoVwf5o%3D&skoid=f5abfce9-8646-42e6-99ea-e8297745b595&sktid=babcc559-b87a-427b-9175-f021a70345c0&skt=2022-04-09T09%3A49%3A47Z&ske=2022-04-10T17%3A59%3A47Z&sks=b&skv=2019-07-07&st=2022-04-09T16%3A07%3A13Z&se=2022-04-10T00%3A17%3A13Z&sp=r',\n",
       "  'azureml-logs/70_driver_log.txt': 'https://azuremlworkspa4632104135.blob.core.windows.net/azureml/ExperimentRun/dcid.diabetes-training_1649521014_99425c33/azureml-logs/70_driver_log.txt?sv=2019-07-07&sr=b&sig=ZFuSbtkK9jAqlxqbwuqcjXm0tWSK1pRsw2gyeHZWV6Q%3D&skoid=f5abfce9-8646-42e6-99ea-e8297745b595&sktid=babcc559-b87a-427b-9175-f021a70345c0&skt=2022-04-09T09%3A49%3A47Z&ske=2022-04-10T17%3A59%3A47Z&sks=b&skv=2019-07-07&st=2022-04-09T16%3A07%3A13Z&se=2022-04-10T00%3A17%3A13Z&sp=r',\n",
       "  'logs/azureml/9620_azureml.log': 'https://azuremlworkspa4632104135.blob.core.windows.net/azureml/ExperimentRun/dcid.diabetes-training_1649521014_99425c33/logs/azureml/9620_azureml.log?sv=2019-07-07&sr=b&sig=pOlioFOP3D%2FJGtIRiJ%2Fbktni3WNp97CXu001KsrRJos%3D&skoid=f5abfce9-8646-42e6-99ea-e8297745b595&sktid=babcc559-b87a-427b-9175-f021a70345c0&skt=2022-04-09T09%3A49%3A47Z&ske=2022-04-10T17%3A59%3A47Z&sks=b&skv=2019-07-07&st=2022-04-09T16%3A07%3A01Z&se=2022-04-10T00%3A17%3A01Z&sp=r'},\n",
       " 'submittedBy': 'Ovo Okpubuluku'}"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from azureml.train.estimator import Estimator\n",
    "from azureml.core import Experiment, Environment, ScriptRunConfig\n",
    "from azureml.widgets import RunDetails\n",
    "\n",
    "\n",
    "myenv = Environment.from_pip_requirements(\"ovo-azureml\", \"requirements.txt\")\n",
    "myenv.python.user_managed_dependencies = True\n",
    "myenv.register(ws)\n",
    "\n",
    "# Create an estimator\n",
    "estimator = ScriptRunConfig(source_directory=training_folder,\n",
    "                            script='diabetes_training.py',\n",
    "                            compute_target='local',\n",
    "                            environment=myenv)\n",
    "\n",
    "# Create an experiment\n",
    "experiment_name = 'diabetes-training'\n",
    "experiment = Experiment(workspace = ws, name = experiment_name)\n",
    "\n",
    "# Run the experiment based on the estimator\n",
    "run = experiment.submit(config=estimator)\n",
    "RunDetails(run_instance=run).show()\n",
    "run.wait_for_completion(show_output=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As with any experiment run, you can use the **RunDetails** widget to view information about the run and get a link to it in Azure Machine Learning studio."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also retrieve the metrics and outputs from the **Run** object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Regularization Rate 0.01\n",
      "Accuracy 0.774\n",
      "AUC 0.848370565699786\n",
      "\n",
      "\n",
      "azureml-logs/60_control_log.txt\n",
      "azureml-logs/70_driver_log.txt\n",
      "logs/azureml/9620_azureml.log\n",
      "outputs/diabetes_model.pkl\n"
     ]
    }
   ],
   "source": [
    "# Get logged metrics\n",
    "metrics = run.get_metrics()\n",
    "for key in metrics.keys():\n",
    "        print(key, metrics.get(key))\n",
    "print('\\n')\n",
    "for file in run.get_file_names():\n",
    "    print(file)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Register the Trained Model\n",
    "\n",
    "Note that the outputs of the experiment include the trained model file (**diabetes_model.pkl**). You can register this model in your Azure Machine Learning workspace, making it possible to track model versions and retrieve them later."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "diabetes_model version: 1\n",
      "\t Training context : Estimator\n",
      "\t AUC : 0.848370565699786\n",
      "\t Accuracy : 0.774\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from azureml.core import Model\n",
    "\n",
    "# Register the model\n",
    "run.register_model(model_path='outputs/diabetes_model.pkl', model_name='diabetes_model',\n",
    "                   tags={'Training context':'Estimator'},\n",
    "                   properties={\n",
    "                                'AUC': run.get_metrics()['AUC'], \n",
    "                               'Accuracy': run.get_metrics()['Accuracy']\n",
    "                               })\n",
    "\n",
    "# List registered models\n",
    "for model in Model.list(ws):\n",
    "    print(model.name, 'version:', model.version)\n",
    "    for tag_name in model.tags:\n",
    "        tag = model.tags[tag_name]\n",
    "        print ('\\t',tag_name, ':', tag)\n",
    "    for prop_name in model.properties:\n",
    "        prop = model.properties[prop_name]\n",
    "        print ('\\t',prop_name, ':', prop)\n",
    "    print('\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create a Parameterized Training Script\n",
    "\n",
    "You can increase the flexibility of your training experiment by adding parameters to your script, enabling you to repeat the same training experiment with different settings. In this case, you'll add a parameter for the regularization rate used by the Logistic Regression algorithm when training the model.\n",
    "\n",
    "Again, lets start by creating a folder for the parameterized script and the training data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'diabetes-training-params\\\\diabetes.csv'"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import os, shutil\n",
    "\n",
    "# Create a folder for the experiment files\n",
    "training_folder = 'diabetes-training-params'\n",
    "os.makedirs(training_folder, exist_ok=True)\n",
    "\n",
    "# Copy the data file into the experiment folder\n",
    "shutil.copy('data/diabetes.csv', os.path.join(training_folder, \"diabetes.csv\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's create a script containing a parameter for the regularization rate hyperparameter."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Writing diabetes-training-params/diabetes_training_params.py\n"
     ]
    }
   ],
   "source": [
    "%%writefile $training_folder/diabetes_training_params.py\n",
    "# Import libraries\n",
    "from azureml.core import Run\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import joblib\n",
    "import os\n",
    "import argparse\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.metrics import roc_auc_score\n",
    "from sklearn.metrics import roc_curve\n",
    "\n",
    "# Get the experiment run context\n",
    "run = Run.get_context()\n",
    "\n",
    "# Set regularization hyperparameter\n",
    "parser = argparse.ArgumentParser()\n",
    "parser.add_argument('--reg_rate', type=float, dest='reg', default=0.01)\n",
    "args = parser.parse_args()\n",
    "reg = args.reg\n",
    "\n",
    "# load the diabetes dataset\n",
    "print(\"Loading Data...\")\n",
    "# load the diabetes dataset\n",
    "diabetes = pd.read_csv('diabetes.csv')\n",
    "\n",
    "# Separate features and labels\n",
    "X, y = diabetes[['Pregnancies','PlasmaGlucose','DiastolicBloodPressure','TricepsThickness','SerumInsulin','BMI','DiabetesPedigree','Age']].values, diabetes['Diabetic'].values\n",
    "\n",
    "# Split data into training set and test set\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.30, random_state=0)\n",
    "\n",
    "# Train a logistic regression model\n",
    "print('Training a logistic regression model with regularization rate of', reg)\n",
    "run.log('Regularization Rate',  np.float(reg))\n",
    "model = LogisticRegression(C=1/reg, solver=\"liblinear\").fit(X_train, y_train)\n",
    "\n",
    "# calculate accuracy\n",
    "y_hat = model.predict(X_test)\n",
    "acc = np.average(y_hat == y_test)\n",
    "print('Accuracy:', acc)\n",
    "run.log('Accuracy', np.float(acc))\n",
    "\n",
    "# calculate AUC\n",
    "y_scores = model.predict_proba(X_test)\n",
    "auc = roc_auc_score(y_test,y_scores[:,1])\n",
    "print('AUC: ' + str(auc))\n",
    "run.log('AUC', np.float(auc))\n",
    "\n",
    "os.makedirs('outputs', exist_ok=True)\n",
    "joblib.dump(value=model, filename='outputs/diabetes_model.pkl')\n",
    "\n",
    "run.complete()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Use a Framework-Specific Estimator\n",
    "\n",
    "You used a generic **Estimator** class to run the training script, but you can also take advantage of framework-specific estimators that include environment definitions for common machine learning frameworks. In this case, you're using Scikit-Learn, so you can use the **SKLearn** estimator. This means that you don't need to specify the **scikit-learn** package in the configuration.\n",
    "\n",
    "> **Note**: Once again, the training experiment uses a new environment; which must be created the first time it is run."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "e24399d53625448c964aa6b26b280e82",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "_UserRunWidget(widget_settings={'childWidgetDisplay': 'popup', 'send_telemetry': False, 'log_level': 'INFO', '…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/aml.mini.widget.v1": "{\"status\": \"Completed\", \"workbench_run_details_uri\": \"https://ml.azure.com/runs/diabetes-training_1649521670_a56fc174?wsid=/subscriptions/8ef9d38c-cd92-4cea-b46c-0527c93d4acb/resourcegroups/azure-ml-sdk-project/workspaces/azureml-workspace&tid=babcc559-b87a-427b-9175-f021a70345c0\", \"run_id\": \"diabetes-training_1649521670_a56fc174\", \"run_properties\": {\"run_id\": \"diabetes-training_1649521670_a56fc174\", \"created_utc\": \"2022-04-09T16:27:49.580478Z\", \"properties\": {\"_azureml.ComputeTargetType\": \"local\", \"ContentSnapshotId\": \"db107c4b-ec3e-4fb0-a56c-f36d712483d9\", \"azureml.git.repository_uri\": \"https://github.com/ovokpus/Azure-Machine-Learning-Project.git\", \"mlflow.source.git.repoURL\": \"https://github.com/ovokpus/Azure-Machine-Learning-Project.git\", \"azureml.git.branch\": \"main\", \"mlflow.source.git.branch\": \"main\", \"azureml.git.commit\": \"d1aabb6aca40b9f8c61269b61b807d44e2309e31\", \"mlflow.source.git.commit\": \"d1aabb6aca40b9f8c61269b61b807d44e2309e31\", \"azureml.git.dirty\": \"True\"}, \"tags\": {}, \"script_name\": null, \"arguments\": null, \"end_time_utc\": \"2022-04-09T16:28:02.972286Z\", \"status\": \"Completed\", \"log_files\": {\"azureml-logs/60_control_log.txt\": \"https://azuremlworkspa4632104135.blob.core.windows.net/azureml/ExperimentRun/dcid.diabetes-training_1649521670_a56fc174/azureml-logs/60_control_log.txt?sv=2019-07-07&sr=b&sig=aOkfQNQ2qrrX6NshM9xQ0%2BY5rXoy6AH%2BuiO3eAF3sVM%3D&skoid=f5abfce9-8646-42e6-99ea-e8297745b595&sktid=babcc559-b87a-427b-9175-f021a70345c0&skt=2022-04-09T09%3A49%3A47Z&ske=2022-04-10T17%3A59%3A47Z&sks=b&skv=2019-07-07&st=2022-04-09T16%3A18%3A03Z&se=2022-04-10T00%3A28%3A03Z&sp=r\", \"azureml-logs/70_driver_log.txt\": \"https://azuremlworkspa4632104135.blob.core.windows.net/azureml/ExperimentRun/dcid.diabetes-training_1649521670_a56fc174/azureml-logs/70_driver_log.txt?sv=2019-07-07&sr=b&sig=gc41ZKqCJz8Fj%2BGrdP3ASrrTiHd%2BCVNVl5NYw8fguh0%3D&skoid=f5abfce9-8646-42e6-99ea-e8297745b595&sktid=babcc559-b87a-427b-9175-f021a70345c0&skt=2022-04-09T09%3A49%3A47Z&ske=2022-04-10T17%3A59%3A47Z&sks=b&skv=2019-07-07&st=2022-04-09T16%3A18%3A03Z&se=2022-04-10T00%3A28%3A03Z&sp=r\", \"logs/azureml/10344_azureml.log\": \"https://azuremlworkspa4632104135.blob.core.windows.net/azureml/ExperimentRun/dcid.diabetes-training_1649521670_a56fc174/logs/azureml/10344_azureml.log?sv=2019-07-07&sr=b&sig=SquHS3sk4i2wvuag2lxmTgPSge4NJlOY9Jkxl4vvMbU%3D&skoid=f5abfce9-8646-42e6-99ea-e8297745b595&sktid=babcc559-b87a-427b-9175-f021a70345c0&skt=2022-04-09T09%3A49%3A47Z&ske=2022-04-10T17%3A59%3A47Z&sks=b&skv=2019-07-07&st=2022-04-09T16%3A17%3A58Z&se=2022-04-10T00%3A27%3A58Z&sp=r\"}, \"log_groups\": [[\"azureml-logs/60_control_log.txt\"], [\"azureml-logs/70_driver_log.txt\"], [\"logs/azureml/10344_azureml.log\"]], \"run_duration\": \"0:00:13\", \"run_number\": \"1649521669\", \"run_queued_details\": {\"status\": \"Completed\", \"details\": null}}, \"child_runs\": [], \"children_metrics\": {}, \"run_metrics\": [{\"name\": \"Regularization Rate\", \"run_id\": \"diabetes-training_1649521670_a56fc174\", \"categories\": [0], \"series\": [{\"data\": [0.001]}]}, {\"name\": \"Accuracy\", \"run_id\": \"diabetes-training_1649521670_a56fc174\", \"categories\": [0], \"series\": [{\"data\": [0.774]}]}, {\"name\": \"AUC\", \"run_id\": \"diabetes-training_1649521670_a56fc174\", \"categories\": [0], \"series\": [{\"data\": [0.8484919647743668]}]}], \"run_logs\": \"2022-04-09 10:27:53,897|azureml|DEBUG|Inputs:: kwargs: {'OutputCollection': True, 'EnableMLflowTracking': True, 'snapshotProject': True}, track_folders: None, deny_list: None, directories_to_watch: ['logs', 'logs/azureml']\\r\\n2022-04-09 10:27:53,898|azureml.history._tracking.PythonWorkingDirectory|DEBUG|Execution target type: none\\r\\n2022-04-09 10:27:53,899|azureml.history._tracking.PythonWorkingDirectory|DEBUG|Failed to import pyspark with error: No module named 'pyspark'\\r\\n2022-04-09 10:27:53,899|azureml.history._tracking.PythonWorkingDirectory.workingdir|DEBUG|Pinning working directory for filesystems: ['pyfs']\\r\\n2022-04-09 10:27:54,562|azureml.core._experiment_method|DEBUG|Trying to register submit_function search, on method <class 'azureml.train.hyperdrive.runconfig.HyperDriveRunConfig'>\\r\\n2022-04-09 10:27:54,563|azureml.core._experiment_method|DEBUG|Registered submit_function search, on method <class 'azureml.train.hyperdrive.runconfig.HyperDriveRunConfig'>\\r\\n2022-04-09 10:27:54,563|azureml.core._experiment_method|DEBUG|Trying to register submit_function search, on method <class 'azureml.train.hyperdrive.runconfig.HyperDriveConfig'>\\r\\n2022-04-09 10:27:54,563|azureml.core._experiment_method|DEBUG|Registered submit_function search, on method <class 'azureml.train.hyperdrive.runconfig.HyperDriveConfig'>\\r\\n2022-04-09 10:27:54,563|azureml.core.run|DEBUG|Adding new factory <function HyperDriveRun._from_run_dto at 0x0000027BBEB00598> for run source hyperdrive\\r\\n2022-04-09 10:27:54,579|azureml.core.run|DEBUG|Adding new factory <function ScriptRun._from_run_dto at 0x0000027BBE913620> for run source azureml.scriptrun\\r\\n2022-04-09 10:27:54,582|azureml.core.run|DEBUG|Identity in use is not set. Falling back to using AMLToken\\r\\n2022-04-09 10:27:54,582|azureml.core.run|DEBUG|Using AMLToken auth for remote run\\r\\n2022-04-09 10:27:54,589|azureml.core.authentication.TokenRefresherDaemon|DEBUG|Starting daemon and triggering first instance\\r\\n2022-04-09 10:27:54,597|azureml._restclient.clientbase|INFO|Created a worker pool for first use\\r\\n2022-04-09 10:27:54,597|azureml.core.authentication|DEBUG|Time to expire 1792794.40289 seconds\\r\\n2022-04-09 10:27:54,597|azureml._restclient.service_context|DEBUG|Created a static thread pool for ServiceContext class\\r\\n2022-04-09 10:27:54,597|azureml._base_sdk_common.service_discovery|DEBUG|Found history service url in environment variable AZUREML_SERVICE_ENDPOINT, history service url: https://westus2.api.azureml.ms.\\r\\n2022-04-09 10:27:54,599|azureml._base_sdk_common.service_discovery|DEBUG|Found history service url in environment variable AZUREML_SERVICE_ENDPOINT, history service url: https://westus2.api.azureml.ms.\\r\\n2022-04-09 10:27:54,599|azureml._base_sdk_common.service_discovery|DEBUG|Found history service url in environment variable AZUREML_SERVICE_ENDPOINT, history service url: https://westus2.api.azureml.ms.\\r\\n2022-04-09 10:27:54,599|azureml._base_sdk_common.service_discovery|DEBUG|Found history service url in environment variable AZUREML_SERVICE_ENDPOINT, history service url: https://westus2.api.azureml.ms.\\r\\n2022-04-09 10:27:54,601|azureml._base_sdk_common.service_discovery|DEBUG|Found history service url in environment variable AZUREML_SERVICE_ENDPOINT, history service url: https://westus2.api.azureml.ms.\\r\\n2022-04-09 10:27:54,601|azureml._base_sdk_common.service_discovery|DEBUG|Found history service url in environment variable AZUREML_SERVICE_ENDPOINT, history service url: https://westus2.api.azureml.ms.\\r\\n2022-04-09 10:27:54,601|azureml._base_sdk_common.service_discovery|DEBUG|Found history service url in environment variable AZUREML_SERVICE_ENDPOINT, history service url: https://westus2.api.azureml.ms.\\r\\n2022-04-09 10:27:54,726|azureml.core._metrics|DEBUG|numpy.float128 is unsupported, expected for windows\\r\\n2022-04-09 10:27:54,748|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.RunClient.get_by_exp_id-async:False|DEBUG|[START]\\r\\n2022-04-09 10:27:54,748|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.RunClient|DEBUG|ClientBase: Calling get_by_exp_id with url /history/v1.0/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.MachineLearningServices/workspaces/{workspaceName}/experimentids/{experimentId}/runs/{runId}\\r\\n2022-04-09 10:27:55,529|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.RunClient.get_by_exp_id-async:False|DEBUG|[STOP]\\r\\n2022-04-09 10:27:55,529|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174|DEBUG|Constructing run from dto. type: azureml.scriptrun, source: None, props: {'_azureml.ComputeTargetType': 'local', 'ContentSnapshotId': 'db107c4b-ec3e-4fb0-a56c-f36d712483d9', 'azureml.git.repository_uri': 'https://github.com/ovokpus/Azure-Machine-Learning-Project.git', 'mlflow.source.git.repoURL': 'https://github.com/ovokpus/Azure-Machine-Learning-Project.git', 'azureml.git.branch': 'main', 'mlflow.source.git.branch': 'main', 'azureml.git.commit': 'd1aabb6aca40b9f8c61269b61b807d44e2309e31', 'mlflow.source.git.commit': 'd1aabb6aca40b9f8c61269b61b807d44e2309e31', 'azureml.git.dirty': 'True'}\\r\\n2022-04-09 10:27:55,530|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunContextManager|DEBUG|Valid logs dir, setting up content loader\\r\\n2022-04-09 10:27:55,530|azureml|WARNING|Could not import azureml.mlflow or azureml.contrib.mlflow mlflow APIs will not run against AzureML services.  Add azureml-mlflow as a conda dependency for the run if this behavior is desired\\r\\n2022-04-09 10:27:55,531|azureml.WorkerPool|DEBUG|[START]\\r\\n2022-04-09 10:27:55,531|azureml.SendRunKillSignal|DEBUG|[START]\\r\\n2022-04-09 10:27:55,531|azureml.RunStatusContext|DEBUG|[START]\\r\\n2022-04-09 10:27:55,531|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunContextManager.RunStatusContext|DEBUG|[START]\\r\\n2022-04-09 10:27:55,531|azureml.MetricsClient|DEBUG|[START]\\r\\n2022-04-09 10:27:55,531|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.MetricsClient|DEBUG|[START]\\r\\n2022-04-09 10:27:55,531|azureml.ContentUploader|DEBUG|[START]\\r\\n2022-04-09 10:27:55,531|azureml._history.utils.context_managers|DEBUG|starting file watcher\\r\\n2022-04-09 10:27:55,532|azureml._history.utils.context_managers.FileWatcher.UploadQueue|DEBUG|[Start]\\r\\n2022-04-09 10:27:55,532|azureml.TrackFolders|DEBUG|[START]\\r\\n2022-04-09 10:27:55,532|azureml.WorkingDirectoryCM|DEBUG|[START]\\r\\n2022-04-09 10:27:55,532|azureml.history._tracking.PythonWorkingDirectory.workingdir|DEBUG|[START]\\r\\n2022-04-09 10:27:55,532|azureml.history._tracking.PythonWorkingDirectory|INFO|Current working dir: C:\\\\Users\\\\OVOOKP~1\\\\AppData\\\\Local\\\\Temp\\\\azureml_runs\\\\diabetes-training_1649521670_a56fc174\\r\\n2022-04-09 10:27:55,532|azureml.history._tracking.PythonWorkingDirectory.workingdir|DEBUG|Calling pyfs\\r\\n2022-04-09 10:27:55,532|azureml.history._tracking.PythonWorkingDirectory.workingdir|DEBUG|Storing working dir for pyfs as C:\\\\Users\\\\OVOOKP~1\\\\AppData\\\\Local\\\\Temp\\\\azureml_runs\\\\diabetes-training_1649521670_a56fc174\\r\\n2022-04-09 10:27:55,542|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.ArtifactsClient.batch_create_empty_artifacts-async:False|DEBUG|[START]\\r\\n2022-04-09 10:27:55,543|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.ArtifactsClient|DEBUG|ClientBase: Calling batch_create_empty_artifacts with url /artifact/v2.0/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.MachineLearningServices/workspaces/{workspaceName}/artifacts/batch/metadata/{origin}/{container}\\r\\n2022-04-09 10:27:55,971|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.ArtifactsClient.batch_create_empty_artifacts-async:False|DEBUG|[STOP]\\r\\n2022-04-09 10:27:56,171|azureml._history.utils.context_managers.FileWatcher|DEBUG|uploading data to container: azureml blob: ExperimentRun/dcid.diabetes-training_1649521670_a56fc174/logs/azureml/10344_azureml.log path: C:\\\\Users\\\\OVOOKP~1\\\\AppData\\\\Local\\\\Temp\\\\azureml_runs\\\\diabetes-training_1649521670_a56fc174\\\\logs\\\\azureml\\\\10344_azureml.log\\r\\n2022-04-09 10:27:56,172|azureml._history.utils.context_managers.FileWatcher.UploadQueue.WorkerPool|DEBUG|submitting future: result\\r\\n2022-04-09 10:27:56,173|azureml._history.utils.context_managers.FileWatcher.UploadQueue.0_result|DEBUG|Using basic handler - no exception handling\\r\\n2022-04-09 10:27:56,173|azureml._history.utils.context_managers.FileWatcher.UploadQueue|DEBUG|Adding task 0_result to queue of approximate size: 0\\r\\n2022-04-09 10:27:56,429|azureml.core.run|DEBUG|Identity in use is not set. Falling back to using AMLToken\\r\\n2022-04-09 10:27:56,429|azureml.core.run|DEBUG|Using AMLToken auth for remote run\\r\\n2022-04-09 10:27:56,429|azureml._restclient.service_context|DEBUG|Access an existing static threadpool for ServiceContext class\\r\\n2022-04-09 10:27:56,430|azureml._base_sdk_common.service_discovery|DEBUG|Found history service url in environment variable AZUREML_SERVICE_ENDPOINT, history service url: https://westus2.api.azureml.ms.\\r\\n2022-04-09 10:27:56,431|azureml._base_sdk_common.service_discovery|DEBUG|Found history service url in environment variable AZUREML_SERVICE_ENDPOINT, history service url: https://westus2.api.azureml.ms.\\r\\n2022-04-09 10:27:56,432|azureml._base_sdk_common.service_discovery|DEBUG|Found history service url in environment variable AZUREML_SERVICE_ENDPOINT, history service url: https://westus2.api.azureml.ms.\\r\\n2022-04-09 10:27:56,432|azureml._base_sdk_common.service_discovery|DEBUG|Found history service url in environment variable AZUREML_SERVICE_ENDPOINT, history service url: https://westus2.api.azureml.ms.\\r\\n2022-04-09 10:27:56,433|azureml._base_sdk_common.service_discovery|DEBUG|Found history service url in environment variable AZUREML_SERVICE_ENDPOINT, history service url: https://westus2.api.azureml.ms.\\r\\n2022-04-09 10:27:56,433|azureml._base_sdk_common.service_discovery|DEBUG|Found history service url in environment variable AZUREML_SERVICE_ENDPOINT, history service url: https://westus2.api.azureml.ms.\\r\\n2022-04-09 10:27:56,433|azureml._base_sdk_common.service_discovery|DEBUG|Found history service url in environment variable AZUREML_SERVICE_ENDPOINT, history service url: https://westus2.api.azureml.ms.\\r\\n2022-04-09 10:27:56,460|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.RunClient.get_by_exp_id-async:False|DEBUG|[START]\\r\\n2022-04-09 10:27:56,461|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.RunClient|DEBUG|ClientBase: Calling get_by_exp_id with url /history/v1.0/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.MachineLearningServices/workspaces/{workspaceName}/experimentids/{experimentId}/runs/{runId}\\r\\n2022-04-09 10:27:56,677|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.RunClient.get_by_exp_id-async:False|DEBUG|[STOP]\\r\\n2022-04-09 10:27:56,678|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174|DEBUG|Constructing run from dto. type: azureml.scriptrun, source: None, props: {'_azureml.ComputeTargetType': 'local', 'ContentSnapshotId': 'db107c4b-ec3e-4fb0-a56c-f36d712483d9', 'azureml.git.repository_uri': 'https://github.com/ovokpus/Azure-Machine-Learning-Project.git', 'mlflow.source.git.repoURL': 'https://github.com/ovokpus/Azure-Machine-Learning-Project.git', 'azureml.git.branch': 'main', 'mlflow.source.git.branch': 'main', 'azureml.git.commit': 'd1aabb6aca40b9f8c61269b61b807d44e2309e31', 'mlflow.source.git.commit': 'd1aabb6aca40b9f8c61269b61b807d44e2309e31', 'azureml.git.dirty': 'True'}\\r\\n2022-04-09 10:27:56,678|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunContextManager|DEBUG|Valid logs dir, setting up content loader\\r\\n2022-04-09 10:27:56,693|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.MetricsClient|DEBUG|Overrides: Max batch size: 50, batch cushion: 5, Interval: 1.\\r\\n2022-04-09 10:27:56,694|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.MetricsClient.PostMetricsBatchV2.PostMetricsBatchV2Daemon|DEBUG|Starting daemon and triggering first instance\\r\\n2022-04-09 10:27:56,694|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.MetricsClient|DEBUG|Used <class 'azureml._common.async_utils.batch_task_queue.BatchTaskQueue'> for use_batch=True.\\r\\n2022-04-09 10:27:56,735|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174|INFO|complete is not setting status for submitted runs.\\r\\n2022-04-09 10:27:56,735|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.MetricsClient.FlushingMetricsClient|DEBUG|[START]\\r\\n2022-04-09 10:27:56,735|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.MetricsClient|DEBUG|Overrides: Max batch size: 50, batch cushion: 5, Interval: 1.\\r\\n2022-04-09 10:27:56,735|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.MetricsClient.PostMetricsBatch.PostMetricsBatchDaemon|DEBUG|Starting daemon and triggering first instance\\r\\n2022-04-09 10:27:56,735|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.MetricsClient|DEBUG|Used <class 'azureml._common.async_utils.batch_task_queue.BatchTaskQueue'> for use_batch=True.\\r\\n2022-04-09 10:27:56,735|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.MetricsClient.PostMetricsBatch.WaitFlushSource:MetricsClient|DEBUG|[START]\\r\\n2022-04-09 10:27:56,736|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.MetricsClient.PostMetricsBatch.WaitFlushSource:MetricsClient|DEBUG|flush timeout 300 is different from task queue timeout 120, using flush timeout\\r\\n2022-04-09 10:27:56,736|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.MetricsClient.PostMetricsBatch.WaitFlushSource:MetricsClient|DEBUG|Waiting 300 seconds on tasks: [].\\r\\n2022-04-09 10:27:56,736|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.MetricsClient.PostMetricsBatch|DEBUG|\\r\\n2022-04-09 10:27:56,736|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.MetricsClient.PostMetricsBatch.WaitFlushSource:MetricsClient|DEBUG|[STOP]\\r\\n2022-04-09 10:27:56,736|azureml.BatchTaskQueueAdd_1_Batches|DEBUG|[Start]\\r\\n2022-04-09 10:27:56,736|azureml.BatchTaskQueueAdd_1_Batches.WorkerPool|DEBUG|submitting future: _handle_batch\\r\\n2022-04-09 10:27:56,736|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.MetricsClient.PostMetricsBatchV2|DEBUG|Batch size 3.\\r\\n2022-04-09 10:27:56,736|azureml._restclient.service_context.WorkerPool|DEBUG|submitting future: _log_batch_v2\\r\\n2022-04-09 10:27:56,737|azureml.BatchTaskQueueAdd_1_Batches.0__handle_batch|DEBUG|Using basic handler - no exception handling\\r\\n2022-04-09 10:27:56,737|azureml.BatchTaskQueueAdd_1_Batches|DEBUG|Adding task 0__handle_batch to queue of approximate size: 0\\r\\n2022-04-09 10:27:56,737|azureml.BatchTaskQueueAdd_1_Batches|DEBUG|[Stop] - waiting default timeout\\r\\n2022-04-09 10:27:56,737|azureml.BatchTaskQueueAdd_1_Batches.WaitFlushSource:BatchTaskQueueAdd_1_Batches|DEBUG|[START]\\r\\n2022-04-09 10:27:56,737|azureml.BatchTaskQueueAdd_1_Batches.WaitFlushSource:BatchTaskQueueAdd_1_Batches|DEBUG|Overriding default flush timeout from None to 120\\r\\n2022-04-09 10:27:56,737|azureml.BatchTaskQueueAdd_1_Batches.WaitFlushSource:BatchTaskQueueAdd_1_Batches|DEBUG|Waiting 120 seconds on tasks: [AsyncTask(0__handle_batch)].\\r\\n2022-04-09 10:27:56,737|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.MetricsClient|DEBUG|Metrics Client: _log_batch_v2 is calling post_run_metrics posting 3 values.\\r\\n2022-04-09 10:27:56,737|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.MetricsClient._post_run_metrics_log_failed_validations-async:False|DEBUG|[START]\\r\\n2022-04-09 10:27:56,737|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.MetricsClient|DEBUG|ClientBase: Calling _post_run_metrics_log_failed_validations with url None\\r\\n2022-04-09 10:27:56,739|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.MetricsClient.PostMetricsBatchV2.0__log_batch_v2|DEBUG|Using basic handler - no exception handling\\r\\n2022-04-09 10:27:56,739|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.MetricsClient.PostMetricsBatchV2|DEBUG|Adding task 0__log_batch_v2 to queue of approximate size: 0\\r\\n2022-04-09 10:27:56,991|azureml.BatchTaskQueueAdd_1_Batches.0__handle_batch.WaitingTask|DEBUG|[START]\\r\\n2022-04-09 10:27:56,991|azureml.BatchTaskQueueAdd_1_Batches.0__handle_batch.WaitingTask|DEBUG|Awaiter is BatchTaskQueueAdd_1_Batches\\r\\n2022-04-09 10:27:56,991|azureml.BatchTaskQueueAdd_1_Batches.0__handle_batch.WaitingTask|DEBUG|[STOP]\\r\\n2022-04-09 10:27:56,991|azureml.BatchTaskQueueAdd_1_Batches|DEBUG|Waiting on task: 0__handle_batch.\\r\\n1 tasks left. Current duration of flush 0.0 seconds.\\r\\n\\r\\n2022-04-09 10:27:56,991|azureml.BatchTaskQueueAdd_1_Batches.WaitFlushSource:BatchTaskQueueAdd_1_Batches|DEBUG|[STOP]\\r\\n2022-04-09 10:27:56,992|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.MetricsClient.PostMetricsBatchV2.WaitFlushSource:MetricsClient|DEBUG|[START]\\r\\n2022-04-09 10:27:56,992|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.MetricsClient.PostMetricsBatchV2.WaitFlushSource:MetricsClient|DEBUG|flush timeout 300 is different from task queue timeout 120, using flush timeout\\r\\n2022-04-09 10:27:56,992|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.MetricsClient.PostMetricsBatchV2.WaitFlushSource:MetricsClient|DEBUG|Waiting 300 seconds on tasks: [AsyncTask(0__log_batch_v2)].\\r\\n2022-04-09 10:27:57,256|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.MetricsClient._post_run_metrics_log_failed_validations-async:False|DEBUG|[STOP]\\r\\n2022-04-09 10:27:57,521|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.MetricsClient.PostMetricsBatchV2.0__log_batch_v2.WaitingTask|DEBUG|[START]\\r\\n2022-04-09 10:27:57,521|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.MetricsClient.PostMetricsBatchV2.0__log_batch_v2.WaitingTask|DEBUG|Awaiter is PostMetricsBatchV2\\r\\n2022-04-09 10:27:57,521|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.MetricsClient.PostMetricsBatchV2.0__log_batch_v2.WaitingTask|DEBUG|[STOP]\\r\\n2022-04-09 10:27:57,521|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.MetricsClient.PostMetricsBatchV2|DEBUG|Waiting on task: 0__log_batch_v2.\\r\\n1 tasks left. Current duration of flush 0.0 seconds.\\r\\nWaiting on task: 0__log_batch_v2.\\r\\n1 tasks left. Current duration of flush 0.2639191150665283 seconds.\\r\\n\\r\\n2022-04-09 10:27:57,521|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.MetricsClient.PostMetricsBatchV2.WaitFlushSource:MetricsClient|DEBUG|[STOP]\\r\\n2022-04-09 10:27:57,521|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.MetricsClient.FlushingMetricsClient|DEBUG|[STOP]\\r\\n2022-04-09 10:27:57,521|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.MetricsClient.wait_on_ingest-async:False|DEBUG|[START]\\r\\n2022-04-09 10:27:57,521|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.MetricsClient|DEBUG|ClientBase: Calling wait_on_ingest with url /history/v1.0/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.MachineLearningServices/workspaces/{workspaceName}/experiments/{experimentName}/runs/{runId}/metricsingest/wait\\r\\n2022-04-09 10:27:57,733|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.MetricsClient.wait_on_ingest-async:False|DEBUG|[STOP]\\r\\n2022-04-09 10:27:57,734|azureml.history._tracking.PythonWorkingDirectory.workingdir|DEBUG|Uploading tracked directories: [], excluding []\\r\\n2022-04-09 10:27:57,734|azureml.history._tracking.PythonWorkingDirectory.workingdir|DEBUG|Calling track for pyfs\\r\\n2022-04-09 10:27:57,972|azureml.history._tracking.PythonWorkingDirectory.workingdir|DEBUG|Calling pyfs\\r\\n2022-04-09 10:27:57,972|azureml.history._tracking.PythonWorkingDirectory|INFO|Current working dir: C:\\\\Users\\\\OVOOKP~1\\\\AppData\\\\Local\\\\Temp\\\\azureml_runs\\\\diabetes-training_1649521670_a56fc174\\r\\n2022-04-09 10:27:57,972|azureml.history._tracking.PythonWorkingDirectory.workingdir|DEBUG|Reverting working dir from C:\\\\Users\\\\OVOOKP~1\\\\AppData\\\\Local\\\\Temp\\\\azureml_runs\\\\diabetes-training_1649521670_a56fc174 to C:\\\\Users\\\\OVOOKP~1\\\\AppData\\\\Local\\\\Temp\\\\azureml_runs\\\\diabetes-training_1649521670_a56fc174\\r\\n2022-04-09 10:27:57,973|azureml.history._tracking.PythonWorkingDirectory|INFO|Working dir is already updated C:\\\\Users\\\\OVOOKP~1\\\\AppData\\\\Local\\\\Temp\\\\azureml_runs\\\\diabetes-training_1649521670_a56fc174\\r\\n2022-04-09 10:27:57,973|azureml.history._tracking.PythonWorkingDirectory.workingdir|DEBUG|[STOP]\\r\\n2022-04-09 10:27:57,973|azureml.WorkingDirectoryCM|DEBUG|[STOP]\\r\\n2022-04-09 10:27:57,973|azureml.history._tracking.PythonWorkingDirectory.workingdir|DEBUG|Uploading tracked directories: ['./outputs'], excluding ['azureml-logs/driver_log']\\r\\n2022-04-09 10:27:57,973|azureml.history._tracking.PythonWorkingDirectory.workingdir|DEBUG|Calling track for pyfs\\r\\n2022-04-09 10:27:57,973|azureml.history._tracking.PythonWorkingDirectory|DEBUG|./outputs exists as directory, uploading..\\r\\n2022-04-09 10:27:57,973|azureml.history._tracking.PythonWorkingDirectory|DEBUG|Found and adding path to upload: ./outputs\\\\diabetes_model.pkl\\r\\n2022-04-09 10:27:57,973|azureml.history._tracking.PythonWorkingDirectory|DEBUG|Paths to upload is ['./outputs\\\\\\\\diabetes_model.pkl'] in dir ./outputs\\r\\n2022-04-09 10:27:57,973|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.ArtifactsClient.upload_files|DEBUG|Overriding default timeout to 300\\r\\n2022-04-09 10:27:57,973|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.ArtifactsClient.upload_files|DEBUG|[Start]\\r\\n2022-04-09 10:27:57,973|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.ArtifactsClient.batch_create_empty_artifacts-async:False|DEBUG|[START]\\r\\n2022-04-09 10:27:57,974|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.ArtifactsClient|DEBUG|ClientBase: Calling batch_create_empty_artifacts with url /artifact/v2.0/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.MachineLearningServices/workspaces/{workspaceName}/artifacts/batch/metadata/{origin}/{container}\\r\\n2022-04-09 10:27:58,301|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.ArtifactsClient.batch_create_empty_artifacts-async:False|DEBUG|[STOP]\\r\\n2022-04-09 10:27:58,301|azureml._restclient.service_context.WorkerPool|DEBUG|submitting future: perform_upload\\r\\n2022-04-09 10:27:58,301|azureml._restclient.clientbase|DEBUG|ClientBase: Calling create_blob_from_stream with url None\\r\\n2022-04-09 10:27:58,302|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.ArtifactsClient.upload_files.0_perform_upload|DEBUG|Using basic handler - no exception handling\\r\\n2022-04-09 10:27:58,302|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.ArtifactsClient.upload_files|DEBUG|Adding task 0_perform_upload to queue of approximate size: 0\\r\\n2022-04-09 10:27:58,302|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.ArtifactsClient.upload_files|DEBUG|[Stop] - waiting default timeout\\r\\n2022-04-09 10:27:58,302|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.ArtifactsClient.upload_files.WaitFlushSource:upload_files|DEBUG|[START]\\r\\n2022-04-09 10:27:58,302|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.ArtifactsClient.upload_files.WaitFlushSource:upload_files|DEBUG|Overriding default flush timeout from None to 300\\r\\n2022-04-09 10:27:58,302|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.ArtifactsClient.upload_files.WaitFlushSource:upload_files|DEBUG|Waiting 300 seconds on tasks: [AsyncTask(0_perform_upload)].\\r\\n2022-04-09 10:27:58,592|azureml._file_utils.upload|DEBUG|Uploaded blob ExperimentRun/dcid.diabetes-training_1649521670_a56fc174/outputs/diabetes_model.pkl with size 964, file size 964.\\r\\n2022-04-09 10:27:58,829|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.ArtifactsClient.upload_files.0_perform_upload.WaitingTask|DEBUG|[START]\\r\\n2022-04-09 10:27:58,829|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.ArtifactsClient.upload_files.0_perform_upload.WaitingTask|DEBUG|Awaiter is upload_files\\r\\n2022-04-09 10:27:58,829|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.ArtifactsClient.upload_files.0_perform_upload.WaitingTask|DEBUG|[STOP]\\r\\n2022-04-09 10:27:58,829|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.ArtifactsClient.upload_files|DEBUG|Waiting on task: 0_perform_upload.\\r\\n1 tasks left. Current duration of flush 0.0 seconds.\\r\\nWaiting on task: 0_perform_upload.\\r\\n1 tasks left. Current duration of flush 0.2620420455932617 seconds.\\r\\n\\r\\n2022-04-09 10:27:58,830|azureml._SubmittedRun#diabetes-training_1649521670_a56fc174.RunHistoryFacade.ArtifactsClient.upload_files.WaitFlushSource:upload_files|DEBUG|[STOP]\\r\\n2022-04-09 10:27:58,830|azureml.TrackFolders|DEBUG|[STOP]\\r\\n2022-04-09 10:27:58,830|azureml._history.utils.context_managers|DEBUG|exiting ContentUploader, waiting for file_watcher to finish upload...\\r\\n2022-04-09 10:27:58,830|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher called finish, setting event\\r\\n2022-04-09 10:27:58,830|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher received exit event, getting current_stat\\r\\n2022-04-09 10:27:58,831|azureml._history.utils.context_managers.FileWatcher.UploadQueue.WorkerPool|DEBUG|submitting future: result\\r\\n2022-04-09 10:27:58,831|azureml._history.utils.context_managers.FileWatcher.UploadQueue.1_result|DEBUG|Using basic handler - no exception handling\\r\\n2022-04-09 10:27:58,831|azureml._history.utils.context_managers.FileWatcher.UploadQueue|DEBUG|Adding task 1_result to queue of approximate size: 1\\r\\n2022-04-09 10:27:58,831|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher retrieved current_stat, will upload to current_stat\\r\\n2022-04-09 10:27:58,832|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,835|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,836|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,837|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,837|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,838|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,839|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,839|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,840|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,840|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,841|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,841|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,842|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,842|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,843|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,844|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,845|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,845|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,846|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,846|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,847|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,848|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,848|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,849|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,849|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,850|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,850|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,851|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,851|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,852|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,853|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,853|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,854|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,854|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,855|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,855|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,856|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,857|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,857|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,858|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,858|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,859|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,859|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,861|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,861|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,862|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,863|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,864|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,864|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,865|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,865|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,866|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,866|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,867|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,867|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,868|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,869|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,869|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,870|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher uploading files to current_stat...\\r\\n2022-04-09 10:27:58,872|azureml._history.utils.context_managers.FileWatcher.UploadQueue.WorkerPool|DEBUG|submitting future: result\\r\\n2022-04-09 10:27:58,872|azureml._history.utils.context_managers.FileWatcher.UploadQueue.2_result|DEBUG|Using basic handler - no exception handling\\r\\n2022-04-09 10:27:58,872|azureml._history.utils.context_managers.FileWatcher.UploadQueue|DEBUG|Adding task 2_result to queue of approximate size: 2\\r\\n2022-04-09 10:27:58,872|azureml._history.utils.context_managers.FileWatcher|DEBUG|FileWatcher finished uploading to current_stat, finishing task queue\\r\\n2022-04-09 10:27:58,872|azureml._history.utils.context_managers.FileWatcher.UploadQueue|DEBUG|[Stop] - waiting default timeout\\r\\n2022-04-09 10:27:58,872|azureml._history.utils.context_managers.FileWatcher.UploadQueue.WaitFlushSource:UploadQueue|DEBUG|[START]\\r\\n2022-04-09 10:27:58,872|azureml._history.utils.context_managers.FileWatcher.UploadQueue.WaitFlushSource:UploadQueue|DEBUG|Overriding default flush timeout from None to 120\\r\\n2022-04-09 10:27:58,873|azureml._history.utils.context_managers.FileWatcher.UploadQueue.WaitFlushSource:UploadQueue|DEBUG|Waiting 120 seconds on tasks: [AsyncTask(0_result), AsyncTask(1_result), AsyncTask(2_result)].\\r\\n2022-04-09 10:27:58,873|azureml._history.utils.context_managers.FileWatcher.UploadQueue.0_result.WaitingTask|DEBUG|[START]\\r\\n2022-04-09 10:27:58,873|azureml._history.utils.context_managers.FileWatcher.UploadQueue.0_result.WaitingTask|DEBUG|Awaiter is UploadQueue\\r\\n2022-04-09 10:27:58,873|azureml._history.utils.context_managers.FileWatcher.UploadQueue.0_result.WaitingTask|DEBUG|[STOP]\\r\\n2022-04-09 10:27:58,873|azureml._history.utils.context_managers.FileWatcher.UploadQueue.1_result.WaitingTask|DEBUG|[START]\\r\\n2022-04-09 10:27:58,873|azureml._history.utils.context_managers.FileWatcher.UploadQueue.1_result.WaitingTask|DEBUG|Awaiter is UploadQueue\\r\\n2022-04-09 10:27:58,873|azureml._history.utils.context_managers.FileWatcher.UploadQueue.1_result.WaitingTask|DEBUG|[STOP]\\r\\n2022-04-09 10:27:59,127|azureml._history.utils.context_managers.FileWatcher.UploadQueue.2_result.WaitingTask|DEBUG|[START]\\r\\n2022-04-09 10:27:59,127|azureml._history.utils.context_managers.FileWatcher.UploadQueue.2_result.WaitingTask|DEBUG|Awaiter is UploadQueue\\r\\n2022-04-09 10:27:59,127|azureml._history.utils.context_managers.FileWatcher.UploadQueue.2_result.WaitingTask|DEBUG|[STOP]\\r\\n2022-04-09 10:27:59,127|azureml._history.utils.context_managers.FileWatcher.UploadQueue|DEBUG|Waiting on task: 2_result.\\r\\n1 tasks left. Current duration of flush 0.0 seconds.\\r\\n\\r\\n2022-04-09 10:27:59,128|azureml._history.utils.context_managers.FileWatcher.UploadQueue.WaitFlushSource:UploadQueue|DEBUG|[STOP]\\r\\n\\nRun is completed.\", \"graph\": {}, \"widget_settings\": {\"childWidgetDisplay\": \"popup\", \"send_telemetry\": false, \"log_level\": \"INFO\", \"sdk_version\": \"1.40.0\"}, \"loading\": false}"
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "{'runId': 'diabetes-training_1649521670_a56fc174',\n",
       " 'target': 'local',\n",
       " 'status': 'Finalizing',\n",
       " 'startTimeUtc': '2022-04-09T16:27:52.372911Z',\n",
       " 'services': {},\n",
       " 'properties': {'_azureml.ComputeTargetType': 'local',\n",
       "  'ContentSnapshotId': 'db107c4b-ec3e-4fb0-a56c-f36d712483d9',\n",
       "  'azureml.git.repository_uri': 'https://github.com/ovokpus/Azure-Machine-Learning-Project.git',\n",
       "  'mlflow.source.git.repoURL': 'https://github.com/ovokpus/Azure-Machine-Learning-Project.git',\n",
       "  'azureml.git.branch': 'main',\n",
       "  'mlflow.source.git.branch': 'main',\n",
       "  'azureml.git.commit': 'd1aabb6aca40b9f8c61269b61b807d44e2309e31',\n",
       "  'mlflow.source.git.commit': 'd1aabb6aca40b9f8c61269b61b807d44e2309e31',\n",
       "  'azureml.git.dirty': 'True'},\n",
       " 'inputDatasets': [],\n",
       " 'outputDatasets': [],\n",
       " 'runDefinition': {'script': 'diabetes_training_params.py',\n",
       "  'command': '',\n",
       "  'useAbsolutePath': False,\n",
       "  'arguments': ['--reg_rate', '0.001'],\n",
       "  'sourceDirectoryDataStore': None,\n",
       "  'framework': 'Python',\n",
       "  'communicator': 'None',\n",
       "  'target': 'local',\n",
       "  'dataReferences': {},\n",
       "  'data': {},\n",
       "  'outputData': {},\n",
       "  'datacaches': [],\n",
       "  'jobName': None,\n",
       "  'maxRunDurationSeconds': 2592000,\n",
       "  'nodeCount': 1,\n",
       "  'instanceTypes': [],\n",
       "  'priority': None,\n",
       "  'credentialPassthrough': False,\n",
       "  'identity': None,\n",
       "  'environment': {'name': 'ovo-azureml',\n",
       "   'version': 'Autosave_2022-04-09T16:14:18Z_5aba36a1',\n",
       "   'python': {'interpreterPath': 'python',\n",
       "    'userManagedDependencies': True,\n",
       "    'condaDependencies': {'channels': ['anaconda', 'conda-forge'],\n",
       "     'dependencies': ['python=3.6.2',\n",
       "      {'pip': ['adal==1.2.7',\n",
       "        'applicationinsights==0.11.10',\n",
       "        'argcomplete==1.12.3',\n",
       "        'argon2-cffi==21.3.0',\n",
       "        'argon2-cffi-bindings==21.2.0',\n",
       "        'async-generator==1.10',\n",
       "        'attrs==21.4.0',\n",
       "        'azure-common==1.1.28',\n",
       "        'azure-core==1.21.1',\n",
       "        'azure-graphrbac==0.61.1',\n",
       "        'azure-mgmt-authorization==0.61.0',\n",
       "        'azure-mgmt-containerregistry==8.2.0',\n",
       "        'azure-mgmt-core==1.3.0',\n",
       "        'azure-mgmt-keyvault==9.3.0',\n",
       "        'azure-mgmt-resource==20.1.0',\n",
       "        'azure-mgmt-storage==19.1.0',\n",
       "        'azure-storage-blob==12.11.0',\n",
       "        'azureml-core==1.40.0',\n",
       "        'azureml-telemetry==1.40.0',\n",
       "        'azureml-train==1.40.0',\n",
       "        'azureml-train-core==1.40.0',\n",
       "        'azureml-train-restclients-hyperdrive==1.40.0',\n",
       "        'azureml-widgets==1.40.0',\n",
       "        'backcall==0.2.0',\n",
       "        'backports.tempfile==1.0',\n",
       "        'backports.weakref==1.0.post1',\n",
       "        'bcrypt==3.2.0',\n",
       "        'bleach==4.1.0',\n",
       "        'certifi==2020.6.20',\n",
       "        'cffi==1.15.0',\n",
       "        'charset-normalizer==2.0.12',\n",
       "        'colorama==0.4.4',\n",
       "        'contextlib2==21.6.0',\n",
       "        'cryptography==36.0.2',\n",
       "        'cycler==0.11.0',\n",
       "        'dataclasses==0.8',\n",
       "        'decorator==5.1.1',\n",
       "        'defusedxml==0.7.1',\n",
       "        'docker==5.0.3',\n",
       "        'entrypoints==0.4',\n",
       "        'fairness-indicators==0.1.2',\n",
       "        'humanfriendly==10.0',\n",
       "        'idna==3.3',\n",
       "        'importlib-metadata==4.8.3',\n",
       "        'ipykernel==5.5.6',\n",
       "        'ipython==7.16.3',\n",
       "        'ipython-genutils==0.2.0',\n",
       "        'ipywidgets==7.7.0',\n",
       "        'isodate==0.6.1',\n",
       "        'jedi==0.17.2',\n",
       "        'jeepney==0.7.1',\n",
       "        'Jinja2==3.0.3',\n",
       "        'jmespath==0.10.0',\n",
       "        'joblib==1.1.0',\n",
       "        'jsonpickle==2.1.0',\n",
       "        'jsonschema==3.2.0',\n",
       "        'jupyter-client==7.1.2',\n",
       "        'jupyter-core==4.9.2',\n",
       "        'jupyterlab-pygments==0.1.2',\n",
       "        'jupyterlab-widgets==1.1.0',\n",
       "        'kiwisolver==1.3.1',\n",
       "        'knack==0.9.0',\n",
       "        'MarkupSafe==2.0.1',\n",
       "        'matplotlib==3.3.4',\n",
       "        'mistune==0.8.4',\n",
       "        'msal==1.17.0',\n",
       "        'msal-extensions==0.3.1',\n",
       "        'msrest==0.6.21',\n",
       "        'msrestazure==0.6.4',\n",
       "        'nbclient==0.5.9',\n",
       "        'nbconvert==6.0.7',\n",
       "        'nbformat==5.1.3',\n",
       "        'ndg-httpsclient==0.5.1',\n",
       "        'nest-asyncio==1.5.5',\n",
       "        'notebook==6.4.10',\n",
       "        'numpy==1.19.5',\n",
       "        'oauthlib==3.2.0',\n",
       "        'packaging==21.3',\n",
       "        'pandas==1.1.5',\n",
       "        'pandocfilters==1.5.0',\n",
       "        'paramiko==2.10.3',\n",
       "        'parso==0.7.1',\n",
       "        'pathspec==0.9.0',\n",
       "        'pickleshare==0.7.5',\n",
       "        'Pillow==8.4.0',\n",
       "        'pkginfo==1.8.2',\n",
       "        'portalocker==2.4.0',\n",
       "        'prometheus-client==0.14.1',\n",
       "        'prompt-toolkit==3.0.29',\n",
       "        'pyasn1==0.4.8',\n",
       "        'pycparser==2.21',\n",
       "        'Pygments==2.11.2',\n",
       "        'PyJWT==2.3.0',\n",
       "        'PyNaCl==1.5.0',\n",
       "        'pyOpenSSL==21.0.0',\n",
       "        'pyparsing==3.0.7',\n",
       "        'pyreadline==2.1',\n",
       "        'pyrsistent==0.18.0',\n",
       "        'PySocks==1.7.1',\n",
       "        'python-dateutil==2.8.2',\n",
       "        'pytz==2022.1',\n",
       "        'pywin32==227',\n",
       "        'pywinpty==1.1.6',\n",
       "        'PyYAML==6.0',\n",
       "        'pyzmq==22.3.0',\n",
       "        'requests==2.27.1',\n",
       "        'requests-oauthlib==1.3.1',\n",
       "        'scikit-learn==0.24.2',\n",
       "        'scipy==1.5.4',\n",
       "        'SecretStorage==3.3.1',\n",
       "        'Send2Trash==1.8.0',\n",
       "        'six==1.16.0',\n",
       "        'sklearn==0.0',\n",
       "        'tabulate==0.8.9',\n",
       "        'tensorflow-metadata==0.30.0',\n",
       "        'terminado==0.12.1',\n",
       "        'testpath==0.6.0',\n",
       "        'threadpoolctl==3.1.0',\n",
       "        'tornado==6.1',\n",
       "        'traitlets==4.3.3',\n",
       "        'typing_extensions==4.1.1',\n",
       "        'urllib3==1.26.7',\n",
       "        'wcwidth==0.2.5',\n",
       "        'webencodings==0.5.1',\n",
       "        'websocket-client==1.3.1',\n",
       "        'widgetsnbextension==3.6.0',\n",
       "        'wincertstore==0.2',\n",
       "        'zipp==3.6.0']},\n",
       "      'pip'],\n",
       "     'name': 'project_environment'},\n",
       "    'baseCondaEnvironment': None},\n",
       "   'environmentVariables': {'EXAMPLE_ENV_VAR': 'EXAMPLE_VALUE'},\n",
       "   'docker': {'baseImage': 'mcr.microsoft.com/azureml/openmpi3.1.2-ubuntu18.04:20220314.v1',\n",
       "    'platform': {'os': 'Linux', 'architecture': 'amd64'},\n",
       "    'baseDockerfile': None,\n",
       "    'baseImageRegistry': {'address': None, 'username': None, 'password': None},\n",
       "    'enabled': False,\n",
       "    'arguments': []},\n",
       "   'spark': {'repositories': [], 'packages': [], 'precachePackages': True},\n",
       "   'inferencingStackVersion': None},\n",
       "  'history': {'outputCollection': True,\n",
       "   'directoriesToWatch': ['logs'],\n",
       "   'enableMLflowTracking': True,\n",
       "   'snapshotProject': True},\n",
       "  'spark': {'configuration': {'spark.app.name': 'Azure ML Experiment',\n",
       "    'spark.yarn.maxAppAttempts': '1'}},\n",
       "  'parallelTask': {'maxRetriesPerWorker': 0,\n",
       "   'workerCountPerNode': 1,\n",
       "   'terminalExitCodes': None,\n",
       "   'configuration': {}},\n",
       "  'amlCompute': {'name': None,\n",
       "   'vmSize': None,\n",
       "   'retainCluster': False,\n",
       "   'clusterMaxNodeCount': None},\n",
       "  'aiSuperComputer': {'instanceType': 'D2',\n",
       "   'imageVersion': 'pytorch-1.7.0',\n",
       "   'location': None,\n",
       "   'aiSuperComputerStorageData': None,\n",
       "   'interactive': False,\n",
       "   'scalePolicy': None,\n",
       "   'virtualClusterArmId': None,\n",
       "   'tensorboardLogDirectory': None,\n",
       "   'sshPublicKey': None,\n",
       "   'sshPublicKeys': None,\n",
       "   'enableAzmlInt': True,\n",
       "   'priority': 'Medium',\n",
       "   'slaTier': 'Standard',\n",
       "   'userAlias': None},\n",
       "  'kubernetesCompute': {'instanceType': None},\n",
       "  'tensorflow': {'workerCount': 1, 'parameterServerCount': 1},\n",
       "  'mpi': {'processCountPerNode': 1},\n",
       "  'pyTorch': {'communicationBackend': 'nccl', 'processCount': None},\n",
       "  'hdi': {'yarnDeployMode': 'Cluster'},\n",
       "  'containerInstance': {'region': None, 'cpuCores': 2.0, 'memoryGb': 3.5},\n",
       "  'exposedPorts': None,\n",
       "  'docker': {'useDocker': False,\n",
       "   'sharedVolumes': True,\n",
       "   'shmSize': '2g',\n",
       "   'arguments': []},\n",
       "  'cmk8sCompute': {'configuration': {}},\n",
       "  'commandReturnCodeConfig': {'returnCode': 'Zero',\n",
       "   'successfulReturnCodes': []},\n",
       "  'environmentVariables': {},\n",
       "  'applicationEndpoints': {},\n",
       "  'parameters': []},\n",
       " 'logFiles': {'azureml-logs/60_control_log.txt': 'https://azuremlworkspa4632104135.blob.core.windows.net/azureml/ExperimentRun/dcid.diabetes-training_1649521670_a56fc174/azureml-logs/60_control_log.txt?sv=2019-07-07&sr=b&sig=5%2BkcP%2FFo%2B9YdiPxrbhGY9rHOyTD0SNujwk6p9NR%2Bop0%3D&skoid=f5abfce9-8646-42e6-99ea-e8297745b595&sktid=babcc559-b87a-427b-9175-f021a70345c0&skt=2022-04-09T09%3A49%3A47Z&ske=2022-04-10T17%3A59%3A47Z&sks=b&skv=2019-07-07&st=2022-04-09T16%3A17%3A58Z&se=2022-04-10T00%3A27%3A58Z&sp=r',\n",
       "  'azureml-logs/70_driver_log.txt': 'https://azuremlworkspa4632104135.blob.core.windows.net/azureml/ExperimentRun/dcid.diabetes-training_1649521670_a56fc174/azureml-logs/70_driver_log.txt?sv=2019-07-07&sr=b&sig=cKmMzVu5sOwRkl3Wy8AHBKndryWCGWlaH8V6KMHwDDg%3D&skoid=f5abfce9-8646-42e6-99ea-e8297745b595&sktid=babcc559-b87a-427b-9175-f021a70345c0&skt=2022-04-09T09%3A49%3A47Z&ske=2022-04-10T17%3A59%3A47Z&sks=b&skv=2019-07-07&st=2022-04-09T16%3A17%3A58Z&se=2022-04-10T00%3A27%3A58Z&sp=r',\n",
       "  'logs/azureml/10344_azureml.log': 'https://azuremlworkspa4632104135.blob.core.windows.net/azureml/ExperimentRun/dcid.diabetes-training_1649521670_a56fc174/logs/azureml/10344_azureml.log?sv=2019-07-07&sr=b&sig=SquHS3sk4i2wvuag2lxmTgPSge4NJlOY9Jkxl4vvMbU%3D&skoid=f5abfce9-8646-42e6-99ea-e8297745b595&sktid=babcc559-b87a-427b-9175-f021a70345c0&skt=2022-04-09T09%3A49%3A47Z&ske=2022-04-10T17%3A59%3A47Z&sks=b&skv=2019-07-07&st=2022-04-09T16%3A17%3A58Z&se=2022-04-10T00%3A27%3A58Z&sp=r'},\n",
       " 'submittedBy': 'Ovo Okpubuluku'}"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from azureml.train.sklearn import SKLearn\n",
    "from azureml.widgets import RunDetails\n",
    "from azureml.core import ScriptRunConfig, Environment\n",
    "\n",
    "\n",
    "myenv = Environment.from_pip_requirements(\"ovo-azureml\", \"requirements.txt\")\n",
    "myenv.python.user_managed_dependencies = True\n",
    "\n",
    "# Create an estimator\n",
    "estimator = ScriptRunConfig(source_directory=training_folder,\n",
    "                            script='diabetes_training_params.py',\n",
    "                            arguments=['--reg_rate', '0.001'],\n",
    "                            compute_target='local',\n",
    "                            environment=myenv)\n",
    "\n",
    "# Create an experiment\n",
    "experiment_name = 'diabetes-training'\n",
    "experiment = Experiment(workspace = ws, name = experiment_name)\n",
    "\n",
    "# Run the experiment\n",
    "run = experiment.submit(config=estimator)\n",
    "\n",
    "# Show the run details while running\n",
    "RunDetails(run).show()\n",
    "run.wait_for_completion()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once again, you can get the metrics and outputs from the run."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Regularization Rate 0.001\n",
      "Accuracy 0.774\n",
      "AUC 0.8484919647743668\n",
      "\n",
      "\n",
      "azureml-logs/60_control_log.txt\n",
      "azureml-logs/70_driver_log.txt\n",
      "logs/azureml/10344_azureml.log\n",
      "outputs/diabetes_model.pkl\n"
     ]
    }
   ],
   "source": [
    "# Get logged metrics\n",
    "metrics = run.get_metrics()\n",
    "for key in metrics.keys():\n",
    "        print(key, metrics.get(key))\n",
    "print('\\n')\n",
    "for file in run.get_file_names():\n",
    "    print(file)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Register A New Version of the Model\n",
    "\n",
    "Now that you've trained a new model, you can register it as a new version in the workspace."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "diabetes_model version: 2\n",
      "\t Training context : Parameterized SKLearn Estimator\n",
      "\t AUC : 0.8484919647743668\n",
      "\t Accuracy : 0.774\n",
      "\n",
      "\n",
      "diabetes_model version: 1\n",
      "\t Training context : Estimator\n",
      "\t AUC : 0.848370565699786\n",
      "\t Accuracy : 0.774\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from azureml.core import Model\n",
    "\n",
    "# Register the model\n",
    "run.register_model(model_path='outputs/diabetes_model.pkl', model_name='diabetes_model',\n",
    "                   tags={'Training context':'Parameterized SKLearn Estimator'},\n",
    "                   properties={'AUC': run.get_metrics()['AUC'], 'Accuracy': run.get_metrics()['Accuracy']})\n",
    "\n",
    "# List registered models\n",
    "for model in Model.list(ws):\n",
    "    print(model.name, 'version:', model.version)\n",
    "    for tag_name in model.tags:\n",
    "        tag = model.tags[tag_name]\n",
    "        print ('\\t',tag_name, ':', tag)\n",
    "    for prop_name in model.properties:\n",
    "        prop = model.properties[prop_name]\n",
    "        print ('\\t',prop_name, ':', prop)\n",
    "    print('\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Clean Up\n",
    "\n",
    "If you've finished exploring, you can close this notebook and shut down your Compute Instance."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}