azure-container-app-with-hugging-face

Project: Create an Azure Container Application with a Hugging Face model for text generation

This project is one of the projects of the MLOps | Machine Learning Operations course available on Coursera. I really recommend this course if you are interested in the MLOps field or Machine Learning Engineering using the most up-to-date tools and best practices.

Instructors GitHub link:

Alfredo Deza

https://github.com/alfredodeza

Noah Gift

https://github.com/noahgift

What is Azure Container App

Serverless infrastructure for deployments. We can easily deploy with scaling a HuggingFace machine-learning model

Azure Container App has the following features included:

• Create secrets
• Setup continuous deployment
• Revisions
• Monitoring

Auto-Scaling Azure Container App

We can use up to 300 applications replicas that'll be deployed in response to a trigger event. We can also use scale rules to determine the type of events that trigger scaling, i.e HTTP concurrent requests and Azure queue

What is Azure Container Registry

Azure Container Registry (ACR) is a fully managed private Docker registry service provided by Microsoft Azure. It stores and manages container images.

Some features of ACR:

• Geo-replication: scale across multiple regions
• Can automatically build, test, push, and deploy images to Azure with Azure Container Registry Tasks

Use the Azure login action with a service principal secret

One requirement for this project is to create an Azure Service Principal that will permit connecting GitHub to Azure Portal

1. Copy your subscription ID. You can find your in this link
2. Open Azure Cloud Shell in the Azure portal or Azure CLI locally.

   AZURE_SUBSCRIPTION_ID=PASTE_YOUR_SUBSCRIPTION_ID_HERE
   az ad sp create-for-rbac --name "CICD" --role contributor --scopes /subscriptions/$AZURE_SUBSCRIPTION_ID  --json-auth

3. Copy the JSON object and save it for now

For more information check this link

Creating a container registry

1. Create a new resource group and give a name to the registry

2. Go to resource and select the Access keys tab

3. Enable the Admin user and copy one of the available passwords

4. Copy the registry repository name as we will use it in the GitHub Actions Yaml file

Below you can check your container image name. We need to use this name in the GitHub Action Yaml file.

az containerapp update -n ${{ env.AZURE_CONTAINER_APP_NAME }} -g ${{ env.AZURE_GROUP_NAME }} --image textgeneration.azurecr.io/mathewsrc/azure-container-app-with-hugging-face:${{ github.sha }}

GitHub Actions secrets

We need to create a last secret for the Azure JSON object that we saved before. I named my secret as AZURE_CREDENTIALS.

Then we can use the AZURE_CREDENTIALS, PASSWORD and USERNAME in GitHub Actions Yaml file:

steps:
      - name: Azure Login
        uses: azure/login@v1
        with:
          creds: ${{ secrets.AZURE_CREDENTIALS }}

with:
          registry: textgeneration.azurecr.io # Registry, i.e Docker Registry (ghcr.io), Azure Registry (azurecr.io)
          username: ${{ secrets.ACR_USERNAME }}
          password: ${{ secrets.ACR_PASSWORD }}

Creating a new container app

In order to create the container we need to follow the steps below:

1. Create a new resource group
2. Give the container app a global unique name
3. Choose a Region (for me Brazil is the closest region)

Click on create button

If everything works fine you will see this output. Now click on the Got to resource button

Setup ingress

We need now to setup ingress for our container as we are using a FAST API app that need a HTTP communication

1. Go to ingress tab

2. Change the Target port to the same port as the one in the Dockerfile EXPOSE 8000 (or any port you choose)

   

3. Click on save button

Edit and deploy container

1. Click on edit and deploy

   

2. Select the container we just created

   

3. Change CPU and RAM settings and click on save

   

if you go back to the containers tab you can see that the settings of the container have changed

After we set up the container app we can update the GitHub Actions Yaml file with the container app name and group name:

env:
  AZURE_CONTAINER_APP_NAME: textgeneration
  AZURE_GROUP_NAME: huggingface

 az config set extension.use_dynamic_install=yes_without_prompt
            az containerapp registry set -n ${{ env.AZURE_CONTAINER_APP_NAME }} -g ${{ env.AZURE_GROUP_NAME }} --server textgeneration.azurecr.io --username ${{ secrets.ACR_USERNAME }} --password ${{ secrets.ACR_PASSWORD }}
            az containerapp update -n ${{ env.AZURE_CONTAINER_APP_NAME }} -g ${{ env.AZURE_GROUP_NAME }} --cpu 2 --memory 4Gi
            az containerapp update -n ${{ env.AZURE_CONTAINER_APP_NAME }} -g ${{ env.AZURE_GROUP_NAME }} --image textgeneration.azurecr.io/mathewsrc/azure-container-app-with-hugging-face:${{ github.sha }}

GitHub Actions manual trigger

We can disable GitHub Actions auto CI/CD by modifying the Yaml file

on:
  # Automatically trigger it when detected changes in repo. Remove comments to enable
  #push:
  #  branches: 
  #    [ main ]

  # Allow manual trigger 
  workflow_dispatch:

Then we can go to the GitHub Actions tab and trigger our app CI/CD workflow

Running

Complete

Then we can go back to Azure Container App and open the application URL

App Home

POST request

First, add docs at the end of the URL to open FASTAPI

Finally, we can call the Hugging Face model

Result