terraform-provider-azurerm_v2.96.0_x5.exe plugin crashed

[Olivier-Beau]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform (and AzureRM Provider) Version

Terraform v1.1.6

• Installed hashicorp/azurerm v2.96.0 (signed by HashiCorp)
• Installed hashicorp/azuread v2.18.0 (signed by HashiCorp)

Affected Resource(s)

datasources (like azurerm_log_analytics_workspace) among others.

Terraform Configuration Files

Crashes started yesterday with no reason.
(No changes done on the configuration files)

Debug Output

lots of "Plugin did not respond" like this

│ Error: Plugin did not respond
│
│ with module.main.data.azurerm_log_analytics_workspace.log_solution,
│ on ..\datasource_infra.tf line 30, in data "azurerm_log_analytics_workspace" "log_solution":
│ 30: data "azurerm_log_analytics_workspace" "log_solution" {
│
│ The plugin encountered an error, and failed to respond to the
│ plugin.(*GRPCProvider).ReadDataSource call. The plugin logs may contain
│ more details.

Panic Output

Stack trace from the terraform-provider-azurerm_v2.96.0_x5.exe plugin:

panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xc0000005 code=0x0 addr=0x0 pc=0x54699d5]

goroutine 730 [running]:
github.com/hashicorp/terraform-provider-azurerm/internal/services/automation.dataSourceAutomationAccountRead(0x68, {0x65ca260, 0xc00135c700})
/opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-azurerm/internal/services/automation/automation_account_data_source.go:71 +0x8f5
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).read(0x6ebcc72, {0x75e3718, 0xc001e78080}, 0xc002f2bf50, {0x65ca260, 0xc00135c700})
/opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-azurerm/vendor/github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema/resource.go:346 +0x178
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).ReadDataApply(0xc00054b880, {0x75e3718, 0xc001e78080}, 0xc004b09b80, {0x65ca260, 0xc00135c700})
/opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-azurerm/vendor/github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema/resource.go:569 +0xf7
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).ReadDataSource(0xc001384888, {0x75e3718, 0xc001e78080}, 0xc009ee1840)
/opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-azurerm/vendor/github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema/grpc_provider.go:1133 +0x3a8
github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.(*server).ReadDataSource(0xc0001ad780, {0x75e37c0, 0xc002f2b470}, 0xc0039e2500)
/opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-azurerm/vendor/github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server/server.go:478 +0x37b
github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_ReadDataSource_Handler({0x6b184a0, 0xc0001ad780}, {0x75e37c0, 0xc002f2b470}, 0xc00309a5a0, 0x0)
/opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-azurerm/vendor/github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:416 +0x170
google.golang.org/grpc.(*Server).processUnaryRPC(0xc0001fefc0, {0x76338b8, 0xc00033ef00}, 0xc003092a20, 0xc0014e4150, 0xbc47150, 0x0)
/opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-azurerm/vendor/google.golang.org/grpc/server.go:1292 +0xc6f
google.golang.org/grpc.(*Server).handleStream(0xc0001fefc0, {0x76338b8, 0xc00033ef00}, 0xc003092a20, 0x0)
/opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-azurerm/vendor/google.golang.org/grpc/server.go:1617 +0xa2a
google.golang.org/grpc.(*Server).serveStreams.func1.2()
/opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-azurerm/vendor/google.golang.org/grpc/server.go:940 +0x98
created by google.golang.org/grpc.(*Server).serveStreams.func1
/opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-azurerm/vendor/google.golang.org/grpc/server.go:938 +0x294

Error: The terraform-provider-azurerm_v2.96.0_x5.exe plugin crashed!

Expected Behaviour

Actual Behaviour

Steps to Reproduce

1. terraform plan

Important Factoids

References

[pavlovmilen]

We are experiencing the same issue since yesterday.
Our tf version and providers are the same 1.1.6 for TF and 2.96.0 for azurerm and 2.18.0 for azuread.
It is affecting our azure windows virtual machine plans.
We have a virtual machine module that is being used by various deployments. The VM module uses log analytics workspace and automation account as a data sources. Its failing on obtaining those data source references.
Same stacktrace as the above

On the Azure side of things if we check at the Activity logs for lets say Log analytics workspace there are numerous events like the below:
Operation name: List Workspace Shared Keys
Error code: ClientClosedRequest
Message: The connection has been closed by the client while the server is still processing its request for 'Microsoft.OperationalInsights'.

We use a service principle that has all the relevant permissions.
The terraform plan where vm module is used worked for us until recently.
Re running the plan on a commit that worked previously is failing now.

Also worth mentioning that we have tried with many older version of Terraform and the providers with the same result
We also have a subnet module that references the same log analytics workspace as a data source. When I trigger a deployment that has only the subnet module and does not use the VM module it is able to complete the plan without issues. This makes me think that is related to virtual machines (windows in our case).

[github-actions]

This functionality has been released in v2.97.0 of the Terraform Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[darren-johnson]

Does anyone know what permissions have changed? This config had been working perfectly for us for over 2 years until this occurred. I got round the problem by changing the SPN permissions on the automation account itself.

[pavlovmilen]

@darren-johnson The underlying issue was missing permission Microsoft.Automation/automationAccounts/listKeys/action: https://docs.microsoft.com/en-us/azure/automation/automation-role-based-access-control#reader-role-access-permissions

[darren-johnson]

@darren-johnson The underlying issue was missing permission Microsoft.Automation/automationAccounts/listKeys/action: https://docs.microsoft.com/en-us/azure/automation/automation-role-based-access-control#reader-role-access-permissions

Looking at the MS docs which were updated the same day I'd say this was a recent change.

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.