Bad Request with remote state in azure blob storage

[scotm-gxg]

I receive an error (link to Gist below) after switching to using terraform_remote_state and storing the state in an azure blob storage,

Terraform Version

Terraform v0.11.3

• provider.azurerm v1.1.1

Affected Resource(s)

• azurerm_app_service

Terraform Configuration Files

main.tf

terraform {
    backend "azurerm" {
        storage_account_name = "gxgterraform"
        container_name = "terraform-state-container"
        access_key = XXXXX
        key = "test_terraform_tfstate"
    }
}

data "terraform_remote_state" "core_api_state" {
    backend = "azurerm"
    config {
        storage_account_name = "gxgterraform"
        container_name = "terraform-state-container"
        access_key = XXXXXXXX
        key = "test_terraform_tfstate"
    }
}

module ai {
    source = "azure-infrastructure"
    aspnetcore_environment = "qa"
    instance_name = "testInstance"
}

azure-infrastructure/main.tf

variable "aspnetcore_environment" {}
variable "instance_name" {}

provider "azurerm" {
}

resource "azurerm_resource_group" "terraform_coreapi" {
    name = "${var.instance_name}_core_api_sg"
    location = "South Central US"
}


resource "azurerm_app_service_plan" "core-api" {
    name = "${var.instance_name}-core-api-asp"
    location = "South Central US"
    resource_group_name = "${azurerm_resource_group.terraform_coreapi.name}"

    sku {
        tier = "Free"
        size = "F1"
    }
}

resource "azurerm_app_service" "core-api-auth" {
    name = "${var.instance_name}-core-api-auth"
    location = "${azurerm_resource_group.terraform_coreapi.location}"
    resource_group_name = "${azurerm_resource_group.terraform_coreapi.name}"
    app_service_plan_id = "${azurerm_app_service_plan.core-api.id}"

    site_config {
        dotnet_framework_version = "v4.0",
        use_32_bit_worker_process = true
    }

    app_settings {
        "ASPNETCORE_ENVIRONMENT"= "${var.aspnetcore_environment}"
    }

}

Debug Output

https://gist.github.com/scotm-gxg/166573305a93a0e4aa50d3fd248c3b0b

Panic Output

no crash log

Expected Behavior

terraform apply should succeed

Actual Behavior

An error occurred when creating the Azure app service referenced in the debug output. Note that this error DOES NOT occur when using the default local state file. This occurred after attempting to switch to a state file stored in an Azure blob account.

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

1. terraform init (success)
2. terraform plan (success)
3. terraform apply (fail)

[scotm-gxg]

so, as a last ditch effort, i completely (manually) deleted the azure resources, and deleted the state file in blob storage. Now I no longer get the error.

[DonEstefan]

Did you rename your module ai {} block during your tests? Maybe just fixing a typo?
This can cause behaviour like the one you describe. For some reason the error messages are misleading in this case. Make sure you use terraform mv after changing module names.

[scotm-gxg]

thanks for the tip. luckily we are just using terraform for automating QA environment creation.

I'm going to close this issue since it hasn't happened again.

[tombuildsstuff]

hey @scotm-gxg

Thanks for opening this issue - apologies for the delayed response here!

So the issue you're seeing here is because when the App Service is being "Created" it's actually finding an existing App Service with the same name in the Resource Group / Subscription and trying to update it.

Whilst Terraform itself doesn't know that the App Service exists yet, the Azure's API's are generally Upserts such that when we "create" the App Service this actually updates the configuration in Azure if an App Service already exists with the same name (and resource group). Whilst that's not ideal - it's a design decision in the way the API's work - but in either case the update error you're seeing is a known issue where the App Service API is broken in Azure. There's a Github issue tracking this here: Azure/azure-rest-api-specs#1697

In the interim, you should be able to Import the existing App Service into the Terraform State, then Terraform will know not to try and update it (providing the configs match). Due to the bug in the API mentioned above, at this time Terraform is only able to delete + recreate the App Service in order to make any changes (it's unfortunate, but there's not a lot else we can do at this time).

Thanks!

[scotm-gxg]

good to know. thanks!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 hashibot-feedback@hashicorp.com. Thanks!