Error from Container Apps- Failed to connect to MSI. Please make sure MSI is configured correctly.

[guitarrapc]

az feedback auto-generates most of the information requested below, as of CLI version 2.37.0

Related command

az login --identity

Describe the bug

Running azure-cli in ContainerApps with MSI, both System assigned and User assigned, but az login --identity failed with 405.

Failed to connect to MSI. Please make sure MSI is configured correctly.
Get Token request returned: <Response [405]>

To Reproduce

1. Prepare ResourceGroup foo.
2. Prepare ContainerApps Environment bar.
3. Save YAML as follows, please replace xxxx with your subscription id.

kind: containerapp
location: japaneast
name: azure-cli
resourceGroup: foo
type: Microsoft.App/containerApps
identity:
  type: "systemAssigned"
properties:
  managedEnvironmentId: "/subscriptions/xxxx/resourceGroups/foo/providers/Microsoft.App/managedEnvironments/bar"
  configuration:
    activeRevisionsMode: Single
  template:
    containers:
      - image: "mcr.microsoft.com/azure-cli"
        name: azure-cli
        command:
          ["/bin/bash", "-c", "while true; do ping localhost; sleep 60;done"]
        resources:
          cpu: 0.25
          memory: 0.5Gi
    scale:
      minReplicas: 1
      maxReplicas: 1

4. Deploy Container App.

az containerapp create -n "azure-cli" -g "foo" --yaml ./deploy/containerapp/azure-cli.yaml

5. (optional) Assign UserAssignedIdentity if needed.

az containerapp identity assign --name "azure-cli" --resource-group "foo" --user-assigned "/subscriptions/xxxxx/resourceGroups/foo/providers/Microsoft.ManagedIdentity/userAssignedIdentities/<YOUR_ID>"

6. Go to console in Container App side Menu.
7. Run az login --identity. You will find error message describe about.

Expected behavior

Managed Identity can be use for az login as describe in doc.

Managed identities in Azure Container Apps | Microsoft Docs

Environment summary

Docker

Additional context

# az --version
azure-cli                         2.37.0

core                              2.37.0
telemetry                          1.0.6

Dependencies:
msal                            1.18.0b1
azure-mgmt-resource             21.1.0b1

Python location '/usr/local/bin/python'
Extensions directory '/root/.azure/cliextensions'

Python (Linux) 3.10.4 (main, Apr 20 2022, 01:09:52) [GCC 10.3.1 20211027]

Legal docs and information: aka.ms/AzureCliLegal


Your CLI is up-to-date.

[yonzhan]

route to CXP team

[RakeshMohanMSFT]

@guitarrapc Thank you for reaching out, we are looking into it.

[ghost]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @macolso.

Issue Details

---

az feedback auto-generates most of the information requested below, as of CLI version 2.37.0

Related command

az login --identity

Describe the bug

Running azure-cli in ContainerApps with MSI, both System assigned and User assigned, but az login --identity failed with 405.

Failed to connect to MSI. Please make sure MSI is configured correctly.
Get Token request returned: <Response [405]>

To Reproduce

1. Prepare ResourceGroup foo.
2. Prepare ContainerApps Environment bar.
3. Save YAML as follows, please replace xxxx with your subscription id.

kind: containerapp
location: japaneast
name: azure-cli
resourceGroup: foo
type: Microsoft.App/containerApps
identity:
  type: "systemAssigned"
properties:
  managedEnvironmentId: "/subscriptions/xxxx/resourceGroups/foo/providers/Microsoft.App/managedEnvironments/bar"
  configuration:
    activeRevisionsMode: Single
  template:
    containers:
      - image: "mcr.microsoft.com/azure-cli"
        name: azure-cli
        command:
          ["/bin/bash", "-c", "while true; do ping localhost; sleep 60;done"]
        resources:
          cpu: 0.25
          memory: 0.5Gi
    scale:
      minReplicas: 1
      maxReplicas: 1

4. Deploy Container App.

az containerapp create -n "azure-cli" -g "foo" --yaml ./deploy/containerapp/azure-cli.yaml

5. (optional) Assign UserAssignedIdentity if needed.

az containerapp identity assign --name "azure-cli" --resource-group "foo" --user-assigned "/subscriptions/xxxxx/resourceGroups/foo/providers/Microsoft.ManagedIdentity/userAssignedIdentities/<YOUR_ID>"

6. Go to console in Container App side Menu.
7. Run az login --identity. You will find error message describe about.

Expected behavior

Managed Identity can be use for az login as describe in doc.

Managed identities in Azure Container Apps | Microsoft Docs

Environment summary

Docker

Additional context

# az --version
azure-cli                         2.37.0

core                              2.37.0
telemetry                          1.0.6

Dependencies:
msal                            1.18.0b1
azure-mgmt-resource             21.1.0b1

Python location '/usr/local/bin/python'
Extensions directory '/root/.azure/cliextensions'

Python (Linux) 3.10.4 (main, Apr 20 2022, 01:09:52) [GCC 10.3.1 20211027]

Legal docs and information: aka.ms/AzureCliLegal


Your CLI is up-to-date.

Author:	guitarrapc
Assignees:	-
Labels:	Service Attention, Container Instances, customer-reported, Auto-Assign, ContainerApp
Milestone:	Backlog

[kabal2010]

I'm also facing this issue and wonder if there has been any resolution for it

[Eviato]

Do we have an update on this? As It's a preferred method to manage authentication instead of using a service principal.

[vturecek]

@yonzhan the issue here is that az login --identity does not correctly detect that it is running inside a Container App, causing it to try connecting to the wrong MSI endpoint. Container Apps uses the same MSI endpoint as App Service, which does not support POST requests:

urllib3.connectionpool: http://localhost:42356 "POST /msi/token HTTP/1.1" 405 0
msrestazure.azure_active_directory: MSI: Retrieving a token from http://localhost:42356/msi/token, with payload {'resource': 'https://management.core.windows.net/'}
msrestazure.azure_active_directory: MSI: Failed to retrieve a token from 'http://localhost:42356/msi/token' with an error of '405 Client Error: Method Not Allowed for url: http://localhost:42356/msi/token'. This could be caused by the MSI extension not yet fully provisioned.
cli.azure.cli.core.auth.adal_authentication: throw requests.exceptions.HTTPError when doing MSIAuthentication: 
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/azure/cli/core/auth/adal_authentication.py", line 75, in set_token
    super().set_token()
  File "/usr/local/lib/python3.10/site-packages/msrestazure/azure_active_directory.py", line 598, in set_token
    self.scheme, _, self.token = get_msi_token(self.resource, self.port, self.msi_conf)
  File "/usr/local/lib/python3.10/site-packages/msrestazure/azure_active_directory.py", line 486, in get_msi_token
    result.raise_for_status()
  File "/usr/local/lib/python3.10/site-packages/requests/models.py", line 953, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 405 Client Error: Method Not Allowed for url: http://localhost:42356/msi/token

This appears to be using the msrestazure-for-python, and sure enough we find that library is attempting to determine the environment it is running in by looking at environment variables with no support for Container Apps. We opened an issue there to get this resolved (Azure/msrestazure-for-python#167), which was closed saying that msrestazure-for-python is deprecated in favor of azure-sdk-for-python. @yonzhan can you please advise which python library this needs to be fixed in?

[jiasli]

@vturecek, your investigation is accurate.

The document for App Service's managed identity endpoint https://learn.microsoft.com/en-us/azure/app-service/overview-managed-identity?tabs=portal%2Chttp#rest-endpoint-reference doesn't even describe MSI_ENDPOINT and MSI_SECRET anymore.

It has been concluded that MSAL will support all variations of managed identity (AzureAD/microsoft-authentication-library-for-python#58), including IDENTITY_ENDPOINT and IDENTITY_HEADER of the new managed identity of App Service. Once MSAL code is ready, we will adopt the latest MSAL.

[maskati]

Workaround is to set the environment variable APPSETTING_WEBSITE_SITE_NAME to any value, which will make the CLI interpret itself as running under App Service, reverting to App Service semantics.

export APPSETTING_WEBSITE_SITE_NAME=DUMMY
az login -i

[malthe]

We're getting intermittent "Connection refused" when az login -i is accessing:

http://localhost:42356/msi/token/?resource=https://management.core.windows.net/&api-version=2017-09-01

EDIT: This seems to be covered by #568.

[kabal2010]

Thanks @maskati
Will keep testing with that login setting and hope no issue pop up, but it'll be nice for the team to resolve this in a proper way for Azure Container App

[anthonychu]

@yonzhan Can you please advise @vturecek on how to proceed with this?

This appears to be using the msrestazure-for-python, and sure enough we find that library is attempting to determine the environment it is running in by looking at environment variables with no support for Container Apps. We opened an issue there to get this resolved (Azure/msrestazure-for-python#167), which was closed saying that msrestazure-for-python is deprecated in favor of azure-sdk-for-python. @yonzhan can you please advise which python library this needs to be fixed in?

[jiasli]

We are working on migrating to MSAL to consume the latest managed identity API version in #25959, but there is currently no ETA yet.

[micheleietri]

Workaround is to set the environment variable APPSETTING_WEBSITE_SITE_NAME to any value, which will make the CLI interpret itself as running under App Service, reverting to App Service semantics.

export APPSETTING_WEBSITE_SITE_NAME=DUMMY
az login -i

tried your fix, but still having issues:

in my script i run

az login --identity -u $env:uamiId

and passing the resource id for the uami, but still getting this error

ERROR: The command failed with an unexpected error. Here is the traceback:
ERROR: "client_id" is the only supported explicit identity option on WebApp

[jaromirmikulik]

Should use client_id, not resourceId. (microsoft/azure-container-apps#502 (comment))

[joeybdub]

Any idea if this is being looked into or has a expected resolve date?

[johschmidt42]

Thanks @maskati, after setting APPSETTING_WEBSITE_SITE_NAME=DUMMY, the Azure Container App Jobs (GitHub self-hosted runners) can now login successfully!

[adeturner]

Just to add with container app jobs for self hosted ADO agents you need to use both and manually set the client id

export APPSETTING_WEBSITE_SITE_NAME=DUMMY
az login --identity -u $MSI_CLIENT_ID

[WhipperSnapperSlapper]

Any updates on this?

It works for me in an Azure Container App by setting:
az login --identity -u $MSI_CLIENT_ID

The container app is setup like this (bicep):

resource userAssignedManagedIdentity 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = {
  name: myAppUserAssignedManagedIdentity
  location: location
}

resource containerApp 'Microsoft.App/containerApps@2024-03-01' = {
  ...
    template: {
      containers: [
        {
          ...
          env: [
            {
              name: 'APPSETTING_WEBSITE_SITE_NAME'
              value: 'DUMMY'
            }            
            {
              name: 'MSI_CLIENT_ID'
              value: userAssignedManagedIdentity.properties.clientId
            }
  ...
}