Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug: When trying to execute from an Azure Virtual Machine the Azapi wants to use the Managed Identity of the Vm not the AZ-CLI creds #213

Closed
1 task done
ghost opened this issue Oct 7, 2024 · 7 comments
Assignees
Labels
Needs: External Changes ⚙️ When an issue/PR requires changes that are outside of the control of this repo Type: Bug 🪲 Something isn't working

Comments

@ghost
Copy link

ghost commented Oct 7, 2024

Is there an existing issue for this?

  • I have searched the existing issues

Infrastrcuture as Code Type? (Required)

terraform

Module Versions (Required)

ALZ PowerShell Module: latest
Accelerator Bootstrap Modules: latest
Terraform Starter Modules: latest
Bicep Starter Modules: x.x.x

When trying to run the deploy-accelerator command from an azure virtual machine the code is trying to use the managed identity of the virtual machine to authenticate and not the provided az-cli details . az account show provides the correct details for the tenant \ subscription however deployment fails stating there is a cross-tenant token issuer problem .

Input arguments of the ALZ-PowerShell-Module (Optional)

No response

Debug Output/Panic Output (Optional)

No response

Expected Behaviour (Required)

Script should have ran and produced required outputs

Actual Behaviour (Required)

Script errors appears to be trying to use token from tenant VM resides in , so I assume it's managed identity .

│ Error: Failed to perform action

│ with data.azapi_resource_action.locations,
│ on main.tf line 12, in data "azapi_resource_action" "locations":
│ 12: data "azapi_resource_action" "locations" {

│ performing action locations of "Resource: (ResourceId "/subscriptions/mysubscriptiontenant" / Api Version
│ "2022-12-01")": GET https://management.azure.com/subscriptions/mysubscriptiontenant/locations
│ --------------------------------------------------------------------------------
│ RESPONSE 401: 401 Unauthorized
│ ERROR CODE: InvalidAuthenticationTokenTenant
│ --------------------------------------------------------------------------------
│ {
│ "error": {
│ "code": "InvalidAuthenticationTokenTenant",
│ "message": "The access token is from the wrong issuer 'https://sts.windows.net/virtualmachinetenantid/'. It must match the tenant 'https://sts.windows.net/mysubscriptiontenant/' associated with this subscription. Please use the authority (URL) 'https://login.windows.net/fmysubscriptiontenant' to get the token. Note, if the subscription is transferred to another tenant there is no impact to the services, but information about new tenant could take time to propagate (up to an hour). If you just transferred your subscription and see this error message, please try back later."
│ }
│ }

Steps to Reproduce (Optional)

No response

Important Factoids (Optional)

No response

References (Optional)

No response

@ghost ghost added Needs: Triage 🔍 Needs triaging by the team Type: Bug 🪲 Something isn't working labels Oct 7, 2024
@jaredfholgate
Copy link
Member

This is a known issue with the azapi provider when running on an Azure VM. You can set the env var $env:ARM_USE_MSI = $false as a workaround for now. Here is the issue, it is fixed in v2.0 of AzAPI: Azure/terraform-provider-azapi#551

We'll update the bootstrap to target v2.0 when it goes GA (imminent), so will leave this issue open to track that.

@jaredfholgate jaredfholgate added Needs: External Changes ⚙️ When an issue/PR requires changes that are outside of the control of this repo and removed Needs: Triage 🔍 Needs triaging by the team labels Oct 10, 2024
@jaredfholgate jaredfholgate self-assigned this Oct 10, 2024
@ghost
Copy link
Author

ghost commented Oct 14, 2024

Have added the env variable and get a different error message ... │ Error: GET https://api.github.com/orgs/mmmm : 401 Bad credentials []

│ with module.github.data.github_organization.alz,
│ on ....\modules\github\data.tf line 1, in data "github_organization" "alz":
│ 1: data "github_organization" "alz" {

I have added a GITHUB_TOKEN which works to my environment vars to no avail .

@jaredfholgate
Copy link
Member

Have added the env variable and get a different error message ... │ Error: GET https://api.github.com/orgs/mmmm : 401 Bad credentials [] │ │ with module.github.data.github_organization.alz, │ on ....\modules\github\data.tf line 1, in data "github_organization" "alz": │ 1: data "github_organization" "alz" { │

I have added a GITHUB_TOKEN which works to my environment vars to no avail .

This is a separate issue. When you refer to GITHUB_TOKEN, we don’t currently support that env var in the accelerator. Did you set your PAT token in the input config file or as an env var input?

@ghost
Copy link
Author

ghost commented Oct 17, 2024

Have added the env variable and get a different error message ... │ Error: GET https://api.github.com/orgs/mmmm : 401 Bad credentials [] │ │ with module.github.data.github_organization.alz, │ on ....\modules\github\data.tf line 1, in data "github_organization" "alz": │ 1: data "github_organization" "alz" { │
I have added a GITHUB_TOKEN which works to my environment vars to no avail .

This is a separate issue. When you refer to GITHUB_TOKEN, we don’t currently support that env var in the accelerator. Did you set your PAT token in the input config file or as an env var input?

Hey Jared thankyou for your response. :)

In answer to your question I tried both methods ! I have found if I re-issue the token with 'God' rights that I am able to continue and use the install > ATM I am unsure which specific priv allowed the module to continue, however the ones defined in the documentation are not enough in this use-case. I was able to use the existing token running on my 'local' machine , but not from the az-vm with $env:ARM_USE_MSI = $false set .

@jaredfholgate
Copy link
Member

Have added the env variable and get a different error message ... │ Error: GET https://api.github.com/orgs/mmmm : 401 Bad credentials [] │ │ with module.github.data.github_organization.alz, │ on ....\modules\github\data.tf line 1, in data "github_organization" "alz": │ 1: data "github_organization" "alz" { │
I have added a GITHUB_TOKEN which works to my environment vars to no avail .

This is a separate issue. When you refer to GITHUB_TOKEN, we don’t currently support that env var in the accelerator. Did you set your PAT token in the input config file or as an env var input?

Hey Jared thankyou for your response. :)

In answer to your question I tried both methods ! I have found if I re-issue the token with 'God' rights that I am able to continue and use the install > ATM I am unsure which specific priv allowed the module to continue, however the ones defined in the documentation are not enough in this use-case. I was able to use the existing token running on my 'local' machine , but not from the az-vm with $env:ARM_USE_MSI = $false set .

Ok, interesting. Our end to end tests run with a token that has the rights set out in the documentation, so wonder what we might be missing. If you find out, please let us know. I'll close this for now given you have a solution.

@joaobo
Copy link

joaobo commented Oct 18, 2024

$env:ARM_USE_MSI = $false

In addition I also had to edit the terraform.tf file by adding use_msi = false at the provider configuration level.

provider "azurerm" {
use_msi = false
subscription_id = var.bootstrap_subscription_id == "" ? null : var.bootstrap_subscription_id
features {
resource_group {
prevent_deletion_if_contains_resources = false
}
}
storage_use_azuread = true
}

provider "azuredevops" {
_ use_msi = false _
personal_access_token = var.azure_devops_personal_access_token
org_service_url = module.azure_devops.organization_url
}

@jaredfholgate
Copy link
Member

$env:ARM_USE_MSI = $false

In addition I also had to edit the terraform.tf file by adding use_msi = false at the provider configuration level.

provider "azurerm" { use_msi = false subscription_id = var.bootstrap_subscription_id == "" ? null : var.bootstrap_subscription_id features { resource_group { prevent_deletion_if_contains_resources = false } } storage_use_azuread = true }

provider "azuredevops" { _ use_msi = false _ personal_access_token = var.azure_devops_personal_access_token org_service_url = module.azure_devops.organization_url }

I don't know why you needed to do this. Only azapi is impacted.

jaredfholgate added a commit that referenced this issue Dec 11, 2024
# Pull Request

## Issue

#213 

## Description

Fix the regions provider version and use the AVM

e2e test run for this branch:
https://github.com/Azure/accelerator-bootstrap-modules/actions/runs/12273987111

## License

By submitting this pull request, I confirm that my contribution is made
under the terms of the projects associated license.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Needs: External Changes ⚙️ When an issue/PR requires changes that are outside of the control of this repo Type: Bug 🪲 Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants