labs/function-calling/function-calling.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# APIM ❤️ OpenAI\n",
    "\n",
    "## Function calling lab\n",
    "![flow](../../images/function-calling.gif)\n",
    "\n",
    "Playground to try the OpenAI [function calling](https://learn.microsoft.com/azure/ai-services/openai/how-to/function-calling?tabs=non-streaming%2Cpython) feature with an Azure Functions API also managed with APIM. \n",
    "\n",
    "At a high level you can break down working with functions into three steps:\n",
    "1. Call the chat completions API with your functions and the user’s input\n",
    "2. Use the model’s response to call your API or function\n",
    "3. Call the chat completions API again, including the response from your function to get a final response\n",
    "\n",
    "### Result\n",
    "![result](result.png)\n",
    "\n",
    "### TOC\n",
    "- [0️⃣ Initialize notebook variables](#0)\n",
    "- [1️⃣ Create the Azure Resource Group](#1)\n",
    "- [2️⃣ Create deployment using 🦾 Bicep](#2)\n",
    "- [3️⃣ Get the deployment outputs](#3)\n",
    "- [4️⃣ Deploy the function](#4)\n",
    "- [🧪 Test the Function API](#function)\n",
    "- [🧪 Test OpenAI function calling](#functioncalling)\n",
    "- [🗑️ Clean up resources](#clean)\n",
    "\n",
    "### Prerequisites\n",
    "- [Python 3.12 or later version](https://www.python.org/) installed\n",
    "- [Pandas Library](https://pandas.pydata.org/) installed\n",
    "- [VS Code](https://code.visualstudio.com/) installed with the [Jupyter notebook extension](https://marketplace.visualstudio.com/items?itemName=ms-toolsai.jupyter) enabled\n",
    "- [Azure CLI](https://learn.microsoft.com/cli/azure/install-azure-cli) installed\n",
    "- [Azure Functions Core Tools](https://learn.microsoft.com/azure/azure-functions/functions-run-local?tabs=windows%2Cisolated-process%2Cnode-v4%2Cpython-v2%2Chttp-trigger%2Ccontainer-apps&pivots=programming-language-python#install-the-azure-functions-core-tools) installed\n",
    "- [An Azure Subscription](https://azure.microsoft.com/free/) with Contributor permissions\n",
    "- [Access granted to Azure OpenAI](https://aka.ms/oai/access) or just enable the mock service\n",
    "- [Sign in to Azure with Azure CLI](https://learn.microsoft.com/cli/azure/authenticate-azure-cli-interactively)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='0'></a>\n",
    "### 0️⃣ Initialize notebook variables\n",
    "\n",
    "- Resources will be suffixed by a unique string based on your subscription id.\n",
    "- The ```mock_webapps``` variable sets the list of deployed Web Apps for the mocking functionality. Clean the ```openai_resources``` list to simulate the OpenAI behaviour with the mocking service.\n",
    "- Adjust the location parameters according your preferences and on the [product availability by Azure region.](https://azure.microsoft.com/explore/global-infrastructure/products-by-region/?cdn=disable&products=cognitive-services,api-management) \n",
    "- Adjust the OpenAI model and version according the [availability by region.](https://learn.microsoft.com/azure/ai-services/openai/concepts/models) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "metadata": {}
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import json\n",
    "import datetime\n",
    "import requests\n",
    "\n",
    "deployment_name = os.path.basename(os.path.dirname(globals()['__vsc_ipynb_file__']))\n",
    "resource_group_name = f\"lab-{deployment_name}\" # change the name to match your naming style\n",
    "resource_group_location = \"westeurope\"\n",
    "apim_resource_name = \"apim\"\n",
    "apim_resource_location = \"westeurope\"\n",
    "apim_resource_sku = \"Basicv2\"\n",
    "openai_resources = [ {\"name\": \"openai1\", \"location\": \"swedencentral\"} ] # list of OpenAI resources to deploy. Clear this list to use only the mock resources\n",
    "openai_resources_sku = \"S0\"\n",
    "openai_model_name = \"gpt-35-turbo\"\n",
    "openai_model_version = \"1106\"\n",
    "openai_deployment_name = \"gpt-35-turbo\"\n",
    "openai_api_version = \"2024-03-01-preview\"\n",
    "openai_specification_url='https://raw.githubusercontent.com/Azure/azure-rest-api-specs/main/specification/cognitiveservices/data-plane/AzureOpenAI/inference/stable/2024-02-01/inference.json'\n",
    "openai_backend_pool = \"openai-backend-pool\"\n",
    "mock_backend_pool = \"mock-backend-pool\"\n",
    "mock_webapps = [ ]\n",
    "\n",
    "log_analytics_name = \"workspace\"\n",
    "app_insights_name = 'insights'\n",
    "\n",
    "function_app_name = \"function\"\n",
    "storage_account_name = \"storage\"\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='1'></a>\n",
    "### 1️⃣ Create the Azure Resource Group\n",
    "All resources deployed in this lab will be created in the specified resource group. Skip this step if you want to use an existing resource group."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "resource_group_stdout = ! az group create --name {resource_group_name} --location {resource_group_location}\n",
    "if resource_group_stdout.n.startswith(\"ERROR\"):\n",
    "    print(resource_group_stdout)\n",
    "else:\n",
    "    print(\"✅ Azure Resource Group \", resource_group_name, \" created ⌚ \", datetime.datetime.now().time())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='2'></a>\n",
    "### 2️⃣ Create deployment using 🦾 Bicep\n",
    "\n",
    "This lab uses [Bicep](https://learn.microsoft.com/azure/azure-resource-manager/bicep/overview?tabs=bicep) to declarative define all the resources that will be deployed. Change the parameters or the [main.bicep](main.bicep) directly to try different configurations. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "if len(openai_resources) > 0:\n",
    "    backend_id = openai_backend_pool if len(openai_resources) > 1 else openai_resources[0].get(\"name\")\n",
    "elif len(mock_webapps) > 0:\n",
    "    backend_id = mock_backend_pool if len(mock_backend_pool) > 1 else mock_webapps[0].get(\"name\")\n",
    "\n",
    "with open(\"policy.xml\", 'r') as policy_xml_file:\n",
    "    policy_template_xml = policy_xml_file.read()\n",
    "    policy_xml = policy_template_xml.replace(\"{backend-id}\", backend_id)\n",
    "    policy_xml_file.close()\n",
    "open(\"policy.xml\", 'w').write(policy_xml)\n",
    "\n",
    "bicep_parameters = {\n",
    "  \"$schema\": \"https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#\",\n",
    "  \"contentVersion\": \"1.0.0.0\",\n",
    "  \"parameters\": {\n",
    "    \"mockWebApps\": { \"value\": mock_webapps },\n",
    "    \"mockBackendPoolName\": { \"value\": mock_backend_pool },\n",
    "    \"openAIBackendPoolName\": { \"value\": openai_backend_pool },\n",
    "    \"openAIConfig\": { \"value\": openai_resources },\n",
    "    \"openAIDeploymentName\": { \"value\": openai_deployment_name },\n",
    "    \"openAISku\": { \"value\": openai_resources_sku },\n",
    "    \"openAIModelName\": { \"value\": openai_model_name },\n",
    "    \"openAIModelVersion\": { \"value\": openai_model_version },\n",
    "    \"openAIAPISpecURL\": { \"value\": openai_specification_url },\n",
    "    \"apimResourceName\": { \"value\": apim_resource_name},\n",
    "    \"apimResourceLocation\": { \"value\": apim_resource_location},\n",
    "    \"apimSku\": { \"value\": apim_resource_sku},\n",
    "    \"logAnalyticsName\": { \"value\": log_analytics_name },\n",
    "    \"applicationInsightsName\": { \"value\": app_insights_name },\n",
    "    \"functionAppName\": { \"value\": function_app_name },\n",
    "    \"storageAccountName\": { \"value\": storage_account_name }\n",
    "  }\n",
    "}\n",
    "with open('params.json', 'w') as bicep_parameters_file:\n",
    "    bicep_parameters_file.write(json.dumps(bicep_parameters))\n",
    "\n",
    "! az deployment group create --name {deployment_name} --resource-group {resource_group_name} --template-file \"main.bicep\" --parameters \"params.json\"\n",
    "\n",
    "open(\"policy.xml\", 'w').write(policy_template_xml)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='3'></a>\n",
    "### 3️⃣ Get the deployment outputs\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "deployment_stdout = ! az deployment group show --name {deployment_name} -g {resource_group_name} --query properties.outputs.apimSubscriptionKey.value -o tsv\n",
    "apim_subscription_key = deployment_stdout.n\n",
    "deployment_stdout = ! az deployment group show --name {deployment_name} -g {resource_group_name} --query properties.outputs.apimResourceGatewayURL.value -o tsv\n",
    "apim_resource_gateway_url = deployment_stdout.n\n",
    "print(\"👉🏻 API Gateway URL: \", apim_resource_gateway_url)\n",
    "\n",
    "deployment_stdout = ! az deployment group show --name {deployment_name} -g {resource_group_name} --query properties.outputs.logAnalyticsWorkspaceId.value -o tsv\n",
    "workspace_id = deployment_stdout.n\n",
    "print(\"👉🏻 Workspace ID: \", workspace_id)\n",
    "\n",
    "deployment_stdout = ! az deployment group show --name {deployment_name} -g {resource_group_name} --query properties.outputs.applicationInsightsAppId.value -o tsv\n",
    "app_id = deployment_stdout.n\n",
    "print(\"👉🏻 App ID: \", app_id)\n",
    "\n",
    "deployment_stdout = ! az deployment group show --name {deployment_name} -g {resource_group_name} --query properties.outputs.functionAppResourceName.value -o tsv\n",
    "function_app_resource_name = deployment_stdout.n\n",
    "print(\"👉🏻 Function Name: \", function_app_resource_name)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='4'></a>\n",
    "### 4️⃣ Deploy the function\n",
    "\n",
    "Deploy the local function project to the function app resource previously created"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "! func azure functionapp publish {function_app_resource_name}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='function'></a>\n",
    "### 🧪 Test the Function API"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "request = { \"location\": \"London\", \"unit\": \"celsius\" }\n",
    "url = apim_resource_gateway_url + \"/weather\"\n",
    "response = requests.post(url, headers = {'api-key':apim_subscription_key}, json = request)\n",
    "if (response.status_code == 200):\n",
    "    data = json.loads(response.text)\n",
    "    print(\"location: \", data.get(\"location\"))\n",
    "    print(\"unit: \", data.get(\"unit\"))\n",
    "    print(\"temperature: \", data.get(\"temperature\"))\n",
    "else:\n",
    "    print(response.text)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='functioncalling'></a>\n",
    "### 🧪 Test OpenAI function calling\n",
    "\n",
    "The following code was reused from the [Azure OpenAI documentation](https://learn.microsoft.com/azure/ai-services/openai/how-to/function-calling?tabs=non-streaming%2Cpython). The `UUID` will be used to search for the end-to-end transaction in Application insights.\n",
    "\n",
    "- Note: run ```pip install openai``` in a terminal before executing this step."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from openai import AzureOpenAI\n",
    "import uuid\n",
    "\n",
    "prompt = \"What's the weather like in London and its big sister cities?\"\n",
    "\n",
    "UUID = str(uuid.uuid4())\n",
    "print(f\"Request-Id: {UUID} - use this ID to trace the requests in Azure Application Insights.\")\n",
    "\n",
    "client = AzureOpenAI(\n",
    "    azure_endpoint=apim_resource_gateway_url,\n",
    "    api_key=apim_subscription_key,\n",
    "    api_version=openai_api_version\n",
    ")\n",
    "\n",
    "# Example function hard coded to return the same weather\n",
    "# the local function calls the API\n",
    "def get_current_weather(location, unit):\n",
    "    request = { \"location\": location, \"unit\": unit }\n",
    "    url = apim_resource_gateway_url + \"/weather\"\n",
    "    response = requests.post(url, headers = {'api-key':apim_subscription_key, \"Request-Id\": UUID}, json = request)\n",
    "    if (response.status_code == 200):\n",
    "        return response.text\n",
    "    else:\n",
    "        return json.dumps({\"location\": location, \"temperature\": \"unknown\"})\n",
    "\n",
    "# Step 1: send the conversation and available functions to the model\n",
    "messages = [{\"role\": \"user\", \"content\": prompt}]\n",
    "tools = [\n",
    "    {\n",
    "        \"type\": \"function\",\n",
    "        \"function\": {\n",
    "            \"name\": \"get_current_weather\",\n",
    "            \"description\": \"Get the current weather in a given location, using the local temperature measuring unit.\",\n",
    "            \"parameters\": {\n",
    "                \"type\": \"object\",\n",
    "                \"properties\": {\n",
    "                    \"location\": {\n",
    "                        \"type\": \"string\",\n",
    "                        \"description\": \"The city and state, e.g. San Francisco, CA\",\n",
    "                    },\n",
    "                    \"unit\": {\n",
    "                        \"type\": \"string\",\n",
    "                        \"description\": \"The temperature measuring unit, e.g. celsius for London or fahrenheit for US cities\",\n",
    "                        \"enum\": [\"celsius\", \"fahrenheit\"]\n",
    "                    },\n",
    "                },\n",
    "                \"required\": [\"location\", \"unit\"],\n",
    "            },\n",
    "        },\n",
    "    }\n",
    "]\n",
    "print(\"▶️ Step 1: start a completion to identify the appropriate functions to invoke based on the prompt\\n\", prompt)\n",
    "response = client.chat.completions.create(\n",
    "    model=openai_deployment_name,\n",
    "    messages=messages,\n",
    "    tools=tools,\n",
    "    tool_choice=\"auto\",  # auto is default, but we'll be explicit\n",
    "    extra_headers = {\"Request-Id\": UUID}\n",
    ")\n",
    "response_message = response.choices[0].message\n",
    "tool_calls = response_message.tool_calls\n",
    "if tool_calls:\n",
    "    # Step 2: call the function\n",
    "    # Note: the JSON response may not always be valid; be sure to handle errors\n",
    "    available_functions = {\n",
    "        \"get_current_weather\": get_current_weather,\n",
    "    }  # only one function in this example, but you can have multiple\n",
    "    messages.append(response_message)  # extend conversation with assistant's reply\n",
    "    # send the info for each function call and function response to the model\n",
    "    print(\"▶️ Step 2: call the functions\")\n",
    "    for tool_call in tool_calls:\n",
    "        function_name = tool_call.function.name\n",
    "        function_to_call = available_functions[function_name]\n",
    "        function_args = json.loads(tool_call.function.arguments)\n",
    "        function_response = function_to_call(\n",
    "            location=function_args.get(\"location\"),\n",
    "            unit=function_args.get(\"unit\"),\n",
    "        )\n",
    "        print(function_response)\n",
    "        messages.append(\n",
    "            {\n",
    "                \"tool_call_id\": tool_call.id,\n",
    "                \"role\": \"tool\",\n",
    "                \"name\": function_name,\n",
    "                \"content\": function_response,\n",
    "            }\n",
    "        )  # extend conversation with function response\n",
    "    print(\"▶️ Step 3: finish with a completion to anwser the user prompt using the function response\")\n",
    "    second_response = client.chat.completions.create(\n",
    "        model=openai_deployment_name,\n",
    "        messages=messages,\n",
    "        extra_headers = {\"Request-Id\": UUID}\n",
    "    )  # get a new response from the model where it can see the function response\n",
    "    print(second_response.choices[0].message.content)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='clean'></a>\n",
    "### 🗑️ Clean up resources\n",
    "\n",
    "When you're finished with the lab, you should remove all your deployed resources from Azure to avoid extra charges and keep your Azure subscription uncluttered.\n",
    "Use the [clean-up-resources notebook](clean-up-resources.ipynb) for that."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}