Provisioning an App Service Plan within a App Service Environment fails

[ghost]

This issue was originally opened by @JoMoore88 as hashicorp/terraform#17477. It was migrated here as a result of the provider split. The original body of the issue is below.

---

Terraform Version

Terraform v0.11.2
Provider.azurerm v1.1.2

Terraform Configuration Files

resource "azurerm_app_service_plan" "ASE_Dev_ASPs" {   
count                         = "${local.devcount}"   
location                      = "${var.location}"   
resource_group_name           = "${azurerm_resource_group.create_resource_group.name}"   
depends_on                    = ["azurerm_template_deployment.ASE_Template_Deployment"]   
name                          = "${var.asename}-ASP-${local.aspnames[count.index]}-DEV"   
properties {   
    app_service_environment_id  = "${  format("/subscriptions/%s/resourceGroups/%s/providers/Microsoft.Web/hostingEnvironments/ASE-%s", "${var.subscription_id}", "${azurerm_resource_group.create_resource_group.name}", "${var.asename}") }"
  }
  sku {
    tier        = "Isolated"
    size        = "${var.aspsize}"
    capacity    = "${var.aspcapacity}"
  }
}

Crash Output

* module.terraform-ase-deployment.azurerm_app_service_plan.ASE_Dev_ASPs[0]: 1 error(s) occurred:

* azurerm_app_service_plan.ASE_Dev_ASPs.0: web.AppServicePlansClient#CreateOrUpdate: Failure sending request: StatusCode=201 -- Original Error: autorest/azure: Service returned an error. Status=201 Code="Unknown" Message="Unknown service error"

Expected Behavior

App service plans should be created, if it already exists the resource should be skipped. This should run error free.

Actual Behavior

The App service plans deploy on the first run; however the error above is generated for any App service plans in the resource block.

Subsequent runs have the App Service Plans detected as new resources, this is because the App Service Plans are not shown in state. I assume due to the above. This is woesome for me as i have another ARM template that runs later on in the project which depends on the azurerm_app_service_plan.ASE_Dev_ASPs block.

Steps to Reproduce

1. Create terraform project that :
   1. Deploys an ASE using ARM in Terraform
   2. Deploys an App Service Plan in terraform that uses the ASE
2. Terraform init
3. Terraform apply

Additional Context

Per the following : https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-manager-async-operations

An asynchronous operation initially returns an HTTP status code of either:

• 201 (Created)
• 202 (Accepted)

When the operation successfully completes, it returns either:

• 200 (OK)
• 204 (No Content)

So it looks like we are getting the status 'created' back but terraform cannot handle said status code. Can this be corrected? Or is there a means of exempting errors/error codes on the HCL side of things that i can implement?

[tombuildsstuff]

Hey @JoMoore88

Thanks for opening this issue

Subsequent runs have the App Service Plans detected as new resources, this is because the App Service Plans are not shown in state

Indeed - in the failure case we don't save anything to the state, since the operator needs to determine how's best to proceed (e.g. if it's ok, the resource can be Imported into Terraform). In this case - the Azure API's returning a HTTP 201 for an internal server error, which as you've seen: a) isn't the status code for an actual http error and b) isn't documented. Out of interest, do you get any more useful information in the logs if you run TF_LOG=1 terraform apply? That'll output the raw http logs which are sent to Azure, which may provide more information

@joshgav 👋 would you be able to reach out to the App Service Team about this? The API returning an internal server error is one thing (and valid) - but returning a 201 "unknown error" feels like the wrong approach?

Thanks!

[miat-asowers]

I'm also seeing this error.

> terraform version
Terraform v0.11.3
+ provider.azurerm v1.1.2

Is a workaround available? (Other than creating the web app / app service manually, of course)

Thanks,
A

[joshgav]

From the docs and the specs it looks the web.AppServicePlansClient#CreateOrUpdate call should only return 200 or 202, so perhaps the SDK lib can't handle the 201.

@naveedaz can you fix the service not to return 201 or update the spec to accept 201's?

@jhendrixMSFT is the missing response code in the spec the reason for this error?

Thanks!

[tombuildsstuff]

@naveedaz any idea of a rough timeframe for fixing this? Thanks!

[tombuildsstuff]

ping @naveedaz :)

[tombuildsstuff]

@joshgav would you be able to reach out to the service team about this? thanks :)

[tombuildsstuff]

@joshgav 👋 - would you be able to reach out to the service team here? :)

[joshgav]

Yes, thanks for the bump! Just in case, @marstr any insights based on your current work with App Service?

[marstr]

Nothing specific to App Service from my side, I would check to see if this definition should have been considered a Long Running Operation. At least for Go updating this would be a breaking change, so even if it gets fixed we probably have to wait for v17 of the Azure SDK for Go to fix it.

I'd be open at looking at hacks in v16 to unblock people though.

[pietro]

@marstr the definition is marked as long running. I think it is missing x-ms-long-running-operation-options and and the definition of the 201 response.

[jadamsowers]

Any update on this? Looks like v17 (and v18) of the Azure SDK for Go have been released.

[tombuildsstuff]

👋 hi all

Just to let you know that a fix for the Application Insights issue mentioned has been released as part of v1.13.0 of the AzureRM Provider :)

Since this fix is unrelated to the original bug report in this issue - I'm going to hide the Application Insights related comments for the moment so that there's no confusion, I hope you don't mind!

Thanks!

[pietro]

I filled Azure/azure-rest-api-specs#3648 and created Azure/azure-rest-api-specs#3774 so hopefully this can get fixed at the Azure API swagger and then pushed to the go SDK which will fix this.

[Phydeauxman]

Any update on this issue? Seems pretty incredible this is still hanging out there. Was reported almost a year ago.

[pietro]

Yeah, we gave up on evaluating ASEs at work because of this.

[miat-asowers]

I'm still hoping that one day I can incorporate ASEs into my Terraform work instead of creating them manually.

[Phydeauxman]

I'm still hoping that one day I can incorporate ASEs into my Terraform work instead of creating them manually.

We have incorporated them into our Terraform code by using an ARM template resource. Works ok but since the actual creation of the ASE takes a long time...once the Terraform deployment times out you need to monitor the creation via the portal. Once the creation completes, you re-run the Terraform deployment again to get the ASE under Terraform control and get any outputs you might have from the ARM template resource.

[tombuildsstuff]

@miat-asowers @Phydeauxman

I'm still hoping that one day I can incorporate ASEs into my Terraform work instead of creating them manually.

Unfortunately ASE's are still blocked on the upstream API issue: Azure/azure-rest-api-specs#4915 - so we're unable to do much with them until that's resolved unfortunately (we're unable to add a Data Source for an existing one, since it'd require writing a test which could provision one, which isn't reasonable possible since the API's broken); as such whilst we'd like to add support for ASE's (and thus be able to poll for them to be successful & then continue so this succeeds; and add test cases covering this) we're blocked on this API issue at this time, unfortunately.

[pietro]

The original issue where provisioning an App Service Plan within a App Service Environment would fail because Azure would respond with HTTP Code 201 and for the SDK it was an invalid code got fixed in the latest release of the SDK (27.0.0).

[Phydeauxman]

Any update on when this will get fixed in the provider now that the SDK issue is resolved?

[tombuildsstuff]

@Phydeauxman this is unfortunately still blocked on the App Service Environment API being broken: Azure/azure-rest-api-specs#4915

[ravulachetan]

@tombuildsstuff;
is the issue blocking creating of ASE or app service plan within a ASE?
i am seeing issues creating app service plan within a ASE.

[logachev]

@ChethanRavulaGit What issues do you see creating app service plan within an ASE?
I was able to deploy it, but it took forever (~30-40 minutes).

I am trying to follow up with the product group on the deployment time issue.

@tombuildsstuff what is a current issue? I was able to deploy ASP inside ASE using Terraform (of course, ASE was pre-created and resource id was harcoded). Having ASE data source is definitely a must have for a good e2e, but I don't think there is any issues with existing ASP resource (assuming deployment time issue is fixed)

[ravulachetan]

@logachev;
I was seeing the similar issue provisioning app service plan within a ASE. It took forever. Initially I thought it was not working but when I let it keep going it did finish after 40-50 minutes. I am right now calling an ARM from terraform to provision app service plan which takes around 3 minutes.

[tombuildsstuff]

👋

We've got tests for provisioning an App Service Plan within an App Service Environment which are now passing - and whilst it does take quite a while to provision (an hour, iirc) it does so successfully and as such I'm going to close this issue for the moment.

Thanks!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.