deploy-to-databox-edge/deploy-to-edge-using-cli.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Copyright (c) Microsoft Corporation. All rights reserved.\n",
    "\n",
    "Licensed under the MIT License."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Deploy Azure ML Hardware Accelerated Models (HAM) to Databox Edge Machine - Using Azure CLI\n",
    "This notebook will walk through creating and deploying a sample client that can test inferencing of an Azure ML HW Accelerated Model. The sample client image contains a script which uses the AzureML SDK to inference and IoT Hub/Device SDK to send messages to your IoT Hub.\n",
    "\n",
    "This notebook will walk you through the following:\n",
    "\n",
    "* [Install dependencies](#installation)\n",
    "* [Configure your Azure CLI environment](#configuration)\n",
    "* [Choose image to deploy](#choose-your-model)\n",
    "* [Create Sample Client image](#sample-client)\n",
    "* [Deploy modules using CLI](#deploy)\n",
    "* [Receive IoT Hub Messages using Event Monitor](#iothub-messages)\n",
    "* [Stop sample client module](#stop-module)\n",
    " \n",
    "## Pre-requisites\n",
    "* An IoT Edge Device (specifically, the Data Box Edge machine with Compute VM enabled)\n",
    "    * Follow [steps 1 - 5a](https://docs.microsoft.com/en-us/azure/databox-online/data-box-edge-deploy-prep) to set up your Data Box Edge machine and enable the Linux Compute VM and an associated IoT Hub. This IoT Hub will deploy the AccelContainerImage to the Compute VM. **Note:** In order to use the Azure ML SDK to deploy your model to your IoT Hub, you must create your Data Box Edge resource in the same subscription as your Azure ML Workspace.\n",
    "* An Azure ML ContainerImage or AccelContainerImage successfully created in your Workspace.\n",
    "    * For an AccelContainerImage, you can follow [this notebook](https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/deployment/accelerated-models/accelerated-models-quickstart.ipynb) through Step 5."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='installation'></a>\n",
    "## Install Azure CLI, IoT extension, and Docker\n",
    "The following code will:\n",
    "- install Azure CLI and [Azure IoT CLI extension](https://github.com/Azure/azure-iot-cli-extension)\n",
    "- install Docker\n",
    "- sign into your Azure account using browser login"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "pip install azure-cli # Install Azure CLI\n",
    "az extension add --name azure-cli-iot-ext # Add IoT CLI extension\n",
    "pip install -U jupyter_console # Fix Jupyter dependencies overridden by Azure CLI installation\n",
    "pip install docker # Install Docker\n",
    "az login # Log into Azure account"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='configuration'></a>\n",
    "## Configure Workspace and IoT information"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we will set your default subscription and resource group for accessing your IoT Hub, assuming your IoT Hub is in the same subscription and resource group as your AzureML Workspace. If not, you should update the iot subscription id and resource group."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.core import Workspace\n",
    "import os\n",
    "\n",
    "ws = Workspace.from_config()\n",
    "\n",
    "# # If config is not local\n",
    "# ws = Workspace(subscription_id=\"\", resource_group=\"\", workspace_name=\"\")\n",
    "# ws.write_config()\n",
    "\n",
    "ws_subscription_id = ws.subscription_id\n",
    "ws_resource_group = ws.resource_group\n",
    "\n",
    "iot_subscription_id = ws.subscription_id # EDIT SUBSCRIPTION ID\n",
    "iot_resource_group = ws.resource_group # EDIT RESOURCE GROUP\n",
    "\n",
    "!az account set --subscription $iot_subscription_id\n",
    "!az configure --defaults group=$iot_resource_group\n",
    "\n",
    "print(\"Workspace information:\")\n",
    "print(ws_subscription_id, ws_resource_group, ws.name, ws.location, sep = '\\n')\n",
    "print(\"------\")\n",
    "print(\"Using following for IoT Hub and Azure CLI:\")\n",
    "print(iot_subscription_id, iot_resource_group, sep = '\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"choose-your-model\"></a>\n",
    "## Configure Azure ML module"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "List all Images in your Azure ML Workspace."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.core.image import Image\n",
    "for i in Image.list(workspace = ws):\n",
    "    print('{}'.format(i.image_location))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Choose a converted image you'd like to deploy by copying the ACR path below. \n",
    "For example, \"workspacename1234.azurecr.io/resnet50.1.brainwaveonnx-image:1\"."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "host_url = \"COPY HERE\"\n",
    "acr_name = host_url.split(\".\")[0]\n",
    "\n",
    "acr_password = !az acr credential show -n $acr_name --query passwords[0].value  --subscription $ws_subscription_id -g $ws_resource_group\n",
    "acr_password = acr_password[0].strip(\"\\\"\")\n",
    "\n",
    "print(\"You are choosing to use the Docker image {} in ACR {}.\".format(host_url, acr_name))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='sample-client'></a>\n",
    "## Configure Sample Client module"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Build sample client image and push to your ACR"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The Docker image built from the /sample-client directory will work with the ResNet50 image created in Quickstart. To create a sample client that works with the Image created from transfer learning, update the line to below. The /sample-client uses ImageNet pictures for inferencing; the /sample-client-tl handles the response differently and uses images of cats and dogs. \n",
    "\n",
    "```\n",
    "client_app = \"sample-client-tl\"\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The default sample client works with the Docker image created from Quickstart\n",
    "client_app = \"sample-client\"\n",
    "version = 1\n",
    "client_url = \"{}.azurecr.io/{}:{}\".format(acr_name, client_app, version)\n",
    "\n",
    "!az acr login --name $acr_name --subscription $ws_subscription_id --resource-group $ws_resource_group\n",
    "!docker build -t $client_url -f ./Dockerfile ./$client_app\n",
    "!docker push $client_url"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Configure Sample Client module\n",
    "The default input and output tensors are: \n",
    "```\n",
    "input_tensor = \"Placeholder:0\"\n",
    "output_tensor = \"classifier/resnet_v1_50/predictions/Softmax:0\"\n",
    "```\n",
    "\n",
    "If you just want to update the input and output tensor names for models other than resnet50 from Quickstart, modify the cell below. \n",
    "\n",
    "For example, the transfer learning notebook would use the same input tensor name and change the output tensor name.\n",
    "```\n",
    "output_tensor = \"classifier_output/Softmax:0\"\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "input_tensor = \"Placeholder:0\"\n",
    "output_tensor = \"classifier/resnet_v1_50/predictions/Softmax:0\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='deploy'></a>\n",
    "## Deploy to Databox Edge Machine Using CLI\n",
    "Here we will utilize the Azure CLI and a pre-defined deployment manifest template which will specify how to deploy your IoT Hub modules. \n",
    "\n",
    "When the Azure ML module first deploys, it will take a few minutes to flash the FPGA with the correct firmware for your model type. While this is happening, the sample client module will output that it is waiting for the flashing to finish.\n",
    "\n",
    "Resources: \n",
    "- [az iot edge set-modules](https://docs.microsoft.com/en-us/azure/iot-edge/how-to-deploy-modules-cli)\n",
    "- [az acr credential show](https://docs.microsoft.com/en-us/cli/azure/acr/credential?view=azure-cli-latest#az-acr-credential-show)\n",
    "- [az iot hub device-identity show-connection-string](https://docs.microsoft.com/en-us/cli/azure/ext/azure-cli-iot-ext/iot/hub/device-identity?view=azure-cli-latest#ext-azure-cli-iot-ext-az-iot-hub-device-identity-show-connection-string)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Provide IoT Hub and IoT Edge Device information\n",
    "When setting up your Compute VM for Databox Edge, you created an IoT Hub and Databox Edge automatically created an IoT Edge device registered to your Compute VM. In order to receive messages in your IoT Hub, copy in your IoT Hub Name and IoT Edge Device name to retrieve your connection string."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# List all iothubs associated with this resource group\n",
    "!az iot hub list --output table"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Copy your IoT Hub name below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Update IoT Hub and Edge Device information\n",
    "hub_name = \"dummy-dbe\"\n",
    "\n",
    "!az iot hub device-identity list --hub-name $hub_name --edge-enabled --output table"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Copy your DeviceId into the variable below\n",
    "device_id = \"computevm\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Deploy\n",
    "#### Create Deployment manifest\n",
    "The below code will replace all the values we've set previously in a newly generated file called deployment.json."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "device_conn_string = !az iot hub device-identity show-connection-string --hub-name $hub_name --device-id $device_id -o tsv\n",
    "device_conn_string = device_conn_string[0]\n",
    "\n",
    "host_status = \"running\"\n",
    "client_status = \"running\"\n",
    "\n",
    "deployment_manifest_path=\"./deployment.json\"\n",
    "\n",
    "# Uses global variables\n",
    "def generate_deployment_manifest():\n",
    "    file = open(\"./deployment_template.json\")\n",
    "    contents = file.read()\n",
    "    contents = contents.replace('$ACR_NAME', acr_name)\n",
    "    contents = contents.replace('$ACR_PASSWORD', acr_password)\n",
    "    contents = contents.replace('$INPUT_TENSOR', input_tensor)\n",
    "    contents = contents.replace('$OUTPUT_TENSOR', output_tensor)\n",
    "    contents = contents.replace('$HOST_URL', host_url)\n",
    "    contents = contents.replace('$CLIENT_URL', client_url)\n",
    "    contents = contents.replace('$DEVICE_CONNECTION_STRING', device_conn_string)\n",
    "    contents = contents.replace('$HOST_STATUS', host_status)\n",
    "    contents = contents.replace('$CLIENT_STATUS', client_status)\n",
    "\n",
    "    with open(deployment_manifest_path, 'wt', encoding='utf-8') as output_file:\n",
    "        output_file.write(contents)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "generate_deployment_manifest()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### AzureML Create Container Options\n",
    "By default, the below is included in the generated deployment.json. If you're using a CPU image you do not need the ``Binds``, ``Privileged``, or ``Devices`` sections and can remove them from the deployment.json file that's generated.\n",
    "```\n",
    "{\n",
    "  \"HostConfig\": {\n",
    "    \"Binds\": [\n",
    "      \"/etc/hosts:/etc/hosts\"\n",
    "    ],\n",
    "    \"Privileged\": true,\n",
    "    \"PortBindings\": {\n",
    "      \"50051/tcp\": [\n",
    "        {\n",
    "          \"HostPort\": \"50051\"\n",
    "        }\n",
    "      ]\n",
    "    },\n",
    "    \"Devices\": [\n",
    "      {\n",
    "        \"PathOnHost\": \"/dev/catapult0\",\n",
    "        \"PathInContainer\": \"/dev/catapult0\"\n",
    "      },\n",
    "      {\n",
    "        \"PathOnHost\": \"/dev/catapult1\",\n",
    "        \"PathInContainer\": \"/dev/catapult1\"\n",
    "      }\n",
    "    ]\n",
    "  }\n",
    "}\n",
    "```\n",
    "#### Sample Client Create Container Options \n",
    "By default, the below is included in the generated deployment.json. You can modify it directly in that file if you want to add other parameters or edit the output tensors to return a list (for SSD-VGG). \n",
    "\n",
    "Additionally, between each inference call is a wait time of 10 seconds to keep the number of messages to your IoT Hub minimal. You can use the --suppress-messages command or change the --wait time between each inference.\n",
    "```\n",
    "{\n",
    "  \"Tty\": true,\n",
    "  \"Cmd\": [\n",
    "    \"--input-tensors\",\n",
    "    \"Placeholder:0\",\n",
    "    \"--output-tensors\",\n",
    "    \"classifier/resnet_v1_50/predictions/Softmax:0\",\n",
    "    \"--wait\",\n",
    "    \"10\"\n",
    "  ]\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Deploy\n",
    "Now we use the Azure IoT CLI to set the deployment manifest for your Data Box Edge IoT Edge device."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# This command sets the deployment.json of your device, which includes the settings for both modules\n",
    "!az iot edge set-modules --device-id $device_id --content $deployment_manifest_path --hub-name $hub_name"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='iothub-messages'></a>\n",
    "## Receive Predictions from IoT Hub\n",
    "The code below uses the CLI to receive messages from IoT Hub. To use the IoT Service Client SDK, read more here: https://docs.microsoft.com/en-us/azure/iot-hub/iot-hub-devguide-messages-read-builtin"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Copy the following command into a bash window:\\n\")\n",
    "print(\"az iot hub monitor-events --hub-name {} --device-id {} --properties app\".format(hub_name, device_id))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='stop-module'></a>\n",
    "## Stop Sample Client Module\n",
    "We can update the desired status of the IoT Hub modules. Here, we will update the deployment.json to specify that the sample client module should be \"stopped\". See the [README.md](./README.md) for more information about how to manipulate the sample client module."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "client_status = \"stopped\"\n",
    "\n",
    "generate_deployment_manifest()\n",
    "\n",
    "# This command sets the deployment.json of your device, which includes the settings for both modules\n",
    "!az iot edge set-modules  --hub-name $hub_name --device-id $device_id --content $deployment_manifest_path"
   ]
  }
 ],
 "metadata": {
  "authors": [
   {
    "name": "paledger"
   }
  ],
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}