how-to-secure-inferencing-vnet.md

Secure an Azure Machine Learning inferencing environment with virtual networks

[!div class="op_single_selector" title1="Select the Azure Machine Learning SDK or CLI version you are using:"]

• SDK/CLI v1
• SDK/CLI v2 (current version)

In this article, you learn how to secure inferencing environments (online endpoints) with a virtual network in Azure Machine Learning. There are two inference options that can be secured using a VNet:

• Azure Machine Learning managed online endpoints
• Azure Kubernetes Service

Tip

This article is part of a series on securing an Azure Machine Learning workflow. See the other articles in this series:

• Virtual network overview
• Secure the workspace resources
• Secure the training environment
• Enable studio functionality
• Use custom DNS
• Use a firewall

For a tutorial on creating a secure workspace, see Tutorial: Create a secure workspace or Tutorial: Create a secure workspace using a template.

Prerequisites

• Read the Network security overview article to understand common virtual network scenarios and overall virtual network architecture.

• An existing virtual network and subnet, that is used to secure the Azure Machine Learning workspace.

• To deploy resources into a virtual network or subnet, your user account must have permissions to the following actions in Azure role-based access control (Azure RBAC):

  • "Microsoft.Network/virtualNetworks/join/action" on the virtual network resource.
  • "Microsoft.Network/virtualNetworks/subnet/join/action" on the subnet resource.

  For more information on Azure RBAC with networking, see the Networking built-in roles.

• If using Azure Kubernetes Service (AKS), you must have an existing AKS cluster secured as described in the Secure Azure Kubernetes Service inference environment article.

Secure managed online endpoints

For information on securing managed online endpoints, see the Use network isolation with managed online endpoints article.

Secure Azure Kubernetes Service online endpoints

To use Azure Kubernetes Service cluster for secure inference, use the following steps:

1. Create or configure a secure Kubernetes inferencing environment.

2. Deploy AzureML extension.

3. Attach the Kubernetes cluster to the workspace.

4. Model deployment with Kubernetes online endpoint can be done using CLI v2, Python SDK v2 and Studio UI.

   • CLI v2 - https://github.com/Azure/azureml-examples/tree/main/cli/endpoints/online/kubernetes
   • Python SDK V2 - https://github.com/Azure/azureml-examples/tree/main/sdk/python/endpoints/online/kubernetes
   • Studio UI - Follow the steps in managed online endpoint deployment through the Studio. After entering the Endpoint name select Kubernetes as the compute type instead of Managed

Limit outbound connectivity from the virtual network

If you don't want to use the default outbound rules and you do want to limit the outbound access of your virtual network, you must allow access to Azure Container Registry. For example, make sure that your Network Security Groups (NSG) contains a rule that allows access to the AzureContainerRegistry.RegionName service tag where `{RegionName} is the name of an Azure region.

Next steps

This article is part of a series on securing an Azure Machine Learning workflow. See the other articles in this series:

• Virtual network overview
• Secure the workspace resources
• Secure the training environment
• Enable studio functionality
• Use custom DNS
• Use a firewall