azuread_service_principal_password: Password not set correctly

[mion00]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform (and AzureAD Provider) Version

Terraform v0.11.13

• provider.azuread v0.2.0

Affected Resource(s)

• azuread_service_principal_password

Terraform Configuration Files

provider "azuread" {
  version = "~> 0.2.0"
}

resource "azuread_application" "test" {
  name = "test"
  available_to_other_tenants = false
}

resource "azuread_service_principal" "test-service-principal" {
  application_id = "${azuread_application.test.application_id}"
}

resource "azuread_service_principal_password" "service-principal-password" {
  service_principal_id = "${azuread_service_principal.test-service-principal.id}"
  value                = "test123"
  end_date             = "2020-01-01T00:00:00Z"
}

Debug Output

Expected Behavior

The service principal is created, and the password for it is set.

Actual Behavior

This bug is the same as the one explained in the issue linked below, but because it was locked I created a new issue here.

Using az CLI, I discovered the following error:

az ad sp credential list --id $(terraform output service_principal)

Parameter 'application_object_id' can not be None.
Traceback (most recent call last):
  File "/opt/az/lib/python3.6/site-packages/knack/cli.py", line 206, in invoke
    cmd_result = self.invocation.execute(args)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 328, in execute
    raise ex
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 386, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 377, in _run_job
    cmd_copy.exception_handler(ex)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/command_modules/role/commands.py", line 69, in graph_err_handler
    raise ex
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 356, in _run_job
    result = cmd_copy(params)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 171, in __call__
    return self.handler(*args, **kwargs)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/__init__.py", line 452, in default_command_handler
    return op(**command_args)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/command_modules/role/custom.py", line 1018, in list_service_principal_credentials
    return _get_service_principal_credentials(graph_client, app_object_id, cert)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/command_modules/role/custom.py", line 1025, in _get_service_principal_credentials
    app_creds = list(graph_client.applications.list_password_credentials(app_object_id))
  File "/opt/az/lib/python3.6/site-packages/msrest/paging.py", line 143, in __next__
    self.advance_page()
  File "/opt/az/lib/python3.6/site-packages/msrest/paging.py", line 129, in advance_page
    self._response = self._get_next(self.next_link)
  File "/opt/az/lib/python3.6/site-packages/azure/graphrbac/operations/applications_operations.py", line 669, in internal_paging
    'applicationObjectId': self._serialize.url("application_object_id", application_object_id, 'str'),
  File "/opt/az/lib/python3.6/site-packages/msrest/serialization.py", line 592, in url
    data = self.validate(data, name, required=True, **kwargs)
  File "/opt/az/lib/python3.6/site-packages/msrest/serialization.py", line 662, in validate
    raise ValidationError("required", name, True)
msrest.exceptions.ValidationError: Parameter 'application_object_id' can not be None.

Steps to Reproduce

1. terraform apply
2. az ad sp credential list --id $(terraform output service_principal)

Important Factoids

References

• azurerm_service_principal_password sometimes fails to assign to a Service Principal terraform-provider-azurerm#2084

[alex0ptr]

I've spent a lot of time today fighting with the same issue. The password for the principal is not set. It's just missing in the UI. The CLI returns the error mentioned above. Using the cli to create the principal (az ad sp create-for-rbac...) it just works. We are on v0.1.0. though.

[MattMencel]

The bug was in Azure-CLI and it looks like the fix is in...

Azure/azure-cli#8900 (comment)

[alex0ptr]

I'm skeptical. The credential was not showing in the UI either as I stated before:

It's just missing in the UI.

[mion00]

Now the az CLI does not give any error, but the password is still not saved correctly after using terraform apply

$ az ad sp credential list --id <generated_id>
[]

I am using the following versions:

• Terraform v0.11.13
• provider.azuread v0.4.0
• az CLI 2.0.68

[arbourd]

I think what's happened is the API has changed. Azure Graph AD v1.6 versus Microsoft Graph v1.0. The UI actually returns different keys for the credentials object:

Azure Graph AD v1.6

"passwordCredentials": [
    {
        "customKeyIdentifier": null,
        "endDate": "2024-05-03T00:00:00Z",
        "keyId": "a1ac2284-48c6-44aa-9003-62b2f5ac421b",
        "startDate": "2019-05-03T09:00:25.9707702Z",
        "value": null
    }
],

Microsoft Graph v1.0

"passwordCredentials": [
    {
        "customKeyIdentifier": null,
        "endDate": "2020-07-10T04:03:30.998Z",
        "keyId": "3bc6171a-a738-448e-b43f-57587abf3e03",
        "startDate": "2019-07-10T04:03:34.987Z",
        "value": "wpHbb3KUKEJ_mZf09+Tg1?:K*V5ZV-f0",
        "createdOn": "2019-07-10T04:03:35.1909766Z",
        "hint": "wpH",
        "displayName": "Password uploaded on Wed Jul 10 2019"
    }
],

Terraform calls the old API that returns a clearly created and attacked password credential:

"passwordCredentials": [
    {
        "customKeyIdentifier" :null,
        "endDate":"2299-12-30T23:00:00Z",
        "keyId":"5b7b5ddb-a7af-ea5c-4f0e-1582a182faa9",
        "startDate":"2019-07-10T03:57:06.4922606Z",
        "value":null
    }
],

[philbal611]

@katbyte Any updates on this issue? Is there anything on the Azure side blocking this functionality?

[arbourd]

@philbal611 I'm pretty sure this is completely Azure blocking at the moment. See this issue: Azure/azure-sdk-for-go#5222

[cbtham]

Is there a workaround or a planned fix for this? It's a major roadblock for creating service principal

[arbourd]

@cbtham Problem appears to be upstream. The SDK doesn't have a work around last time I checked. I'm creating SPs with the azure-cli in Terraform right now.

[cbtham]

how do you do that? Do you have a reference? Let me know if it works for you

[arbourd]

@cbtham I am using a local-exec provisioner to run the CLI commands. It's not pretty.

[dekimsey]

I believe this may be related, but we ran into an issue with destroying the sp password. Though this happened in Terraform, I suspect the same underlying issue is at heart. In our case it appears the Application ownership do not extend to the service principal passwords created in this manner. I created the Application and the SP entries and assigned my coworker ownership of the application, but my co-worker was unable to destroy the SP.

Error: Error removing Password "4ee49bd7-2668-a195-2fb7-c559705ae7a1" from Service Principal "21a3877b-2e43-4829-a1d2-f97d39f6b006": graphrbac.ServicePrincipalsClient#UpdatePasswordCredentials: Failure responding to request: StatusCode=403 -- Original Error: autorest/azure: Service returned an error. Status=403 Code="Unknown" Message="Unknown service error" Details=[{"odata.error":{"code":"Authorization_RequestDenied","date":"2019-10-03T16:22:49","message":{"lang":"en","value":"Insufficient privileges to complete the operation."},"requestId":"3812e550-768e-4ad5-b452-cea2fd233f9b"}}]

Which looks sane according the az ad sp list output.

    "objectId": "21a3877b-2e43-4829-a1d2-f97d39f6b006",
    "objectType": "ServicePrincipal",
    "odata.type": "Microsoft.DirectoryServices.ServicePrincipal",
    "passwordCredentials": [
      {
        "additionalProperties": null,
        "customKeyIdentifier": null,
        "endDate": "2020-08-28T17:34:20.691547+00:00",
        "keyId": "4ee49bd7-2668-a195-2fb7-c559705ae7a1",
        "startDate": "2019-08-28T17:34:20.971457+00:00",
        "value": null
      }
    ],

[cbtham]

So at the moment there is still no fix scheduled?

[dekimsey]

@cbtham, I believe the issue is blocked by an upstream Azure SDK bug. See https://github.com/Azure/azure-sdk-for-go/issues/5222. I'm sure an upvote on the issue could help or poke your Microsoft rep.

[poddm]

I was able to work around this using the deprecated azurerm_azuread_service_principal and azurerm_azuread_service_principal_password resources.

[myrah]

@poddm, which azuread provider version did you use? I tried with v0.4 and v0.6, using deprecated azurerm_azuread_service_principal and azurerm_azuread_service_principal_password, doesn't work, even with additional deprecated azurerm_azuread_application, still no application password was created.

[poddm]

@myrah, it's the deprecated resources in the azurerm provider.

• https://www.terraform.io/docs/providers/azurerm/r/azuread_service_principal_password.html
• https://www.terraform.io/docs/providers/azurerm/r/azuread_service_principal.html

I'm using the latest azurerm provider
provider "azurerm" { version = "~> 1.35.0" }

[lstyles]

This works for me:

resource "azuread_application" "main" {
  name                       = var.sp_name
  available_to_other_tenants = false
}

resource "azuread_service_principal" "main" {
  application_id = azuread_application.main.application_id
}

resource "random_password" "main" {
  length  = 32
  special = false
}

resource "azuread_service_principal_password" "main" {
  service_principal_id = azuread_service_principal.main.id  
  value                = random_password.main.result
  end_date             = "2299-12-31T00:00:00Z"
}

I then use it to create a kubernetes cluster:

...
service_principal {
    client_id = azuread_service_principal.main.application_id
    client_secret = random_password.main.result
  }

depends_on = [
    azuread_service_principal.main,
    azuread_service_principal_password.main
  ]
...

In the portal, I don't see a client secret against the application but the Kubernetes cluster deploys successfully.

Interestingly, I had to add depends_on for azuread_service_principal.main despite it being referenced in kubernetes resource.

I had the same problem as the person who originally raised the issue but upgrading Azure CLI has resolved it for me.

Edit: After further investigation, the reason why the secret isn't showing in the Azure portal is because those are the application secrets and not service principal secrets.

I have just tried this:

resource "azuread_application" "main" {
  name                       = var.sp_name
  available_to_other_tenants = false
}

resource "azuread_service_principal" "main" {
  application_id = azuread_application.main.application_id
}

resource "random_password" "main" {
  length  = 32
  special = false
}

resource "azuread_application_password" "main" {
  application_object_id  = azuread_application.main.id  
  value                  = random_password.main.result
  end_date               = "2299-12-31T00:00:00Z"
}

and then this, in the kubernetes cluster definition:

service_principal {
    client_id = azuread_service_principal.main.application_id
    client_secret = azuread_application_password.main.value
  }

and it works fine. The secret is also showing in the portal. In fact, this is probably the better way to do it as it allows for importing of clusters created via the portal into TF.

Using:
azurerm = "=1.36.1"
azuread = "=0.6.0"

[drdamour]

you can NOT see service principal passwords in the portal AFAIK, only application secrets/passwords. they are slightly different in a single tenant app scenario and WAAAAY different in the multi tenant scenario

[manicminer]

I believe this is a portal usability issue. You can no longer view secrets for service principals in the portal, only secrets for applications.

az ad sp create-for-rbac might not be doing entirely what you expect. It does several things including registering an application, creating a secret for that application and creating an associated service principal - accordingly if you inspect the application in the portal you can see the result.

As @drdamour mentioned, SP passwords and app passwords are somewhat different yet can be used interchangably in some scenarios.

Closing as this is not really related to the provider, however please feel free to comment if there's a subtlety I have overlooked!

[gvilarino]

@manicminer would you elaborate on that please? I am able to see secrets for principals (app registrations). What I'm never able to see after principal creation-via-cli is the principal password (which acts as a secret but it's never shown after that, and you can never see it from the portal).

Thanks

[manicminer]

Hey @gvilarino, it can get confusing with the interchangeable language used in the CLI and elsewhere, but app registrations and service principals (aka enterprise applications) are two different objects in Azure AD. The portal exposes a UI for listing secrets (passwords) for app registrations, but not for service principal secrets. It used to be the case that secrets were stored with the SP, but they are now [typically] stored with the app registrations, and in many auth scenarios you can use a secret from either entity when authenticating with the clientID of the app registration.

In the provider, we have resources for setting either of the two secret types. If you use the azuread_service_principal_password resource, you won’t see it in the Secrets pane of the App Registrations blade in portal as it’s saved with the service principal. For that you can use the azuread_application_password resource.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 hashibot-feedback@hashicorp.com. Thanks!