RC api returning incorrect response when instance already exists

[ocofaigh]

Project Golden Eye

IBM Cloud secrets-manager only allows one secrets manager instance per account by default. When you try to provision another one using terraform, you get back the following error:

╷
│ Error: Error when creating resource instance: Unexpected HTTP status code 503 (503 Service Unavailable) with resp code: null
│ 
│ 
│   with ibm_resource_instance.secrets_manager,
│   on main.tf line 18, in resource "ibm_resource_instance" "secrets_manager":
│   18: resource "ibm_resource_instance" "secrets_manager" {
│ 
╵

The secrets-manager team have said that this response does not come from secrets-manager. When I try in the UI, I get this:

Is this error message Unexpected HTTP status code 503 (503 Service Unavailable) with resp code: null coming from the ibm terraform provider? If so, can we not just return the same error as the secrets-manager api does?

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform IBM Provider Version

$ terraform -v
Terraform v1.0.8
on darwin_amd64
+ provider registry.terraform.io/ibm-cloud/ibm v1.31.0

Affected Resource(s)

• ibm_resource_instance.secrets_manager

Terraform Configuration Files

##################################################################
## Variables
##################################################################
variable "ibmcloud_api_key" {
  type        = string
  description = "The IBM Cloud API Token"
  sensitive   = true
}

variable "prefix" {
  type        = string
  description = "Prefix for name of all resource created by this example"
  default     = "temp-conall"
}

##################################################################
## Providers
##################################################################
terraform {
  required_version = ">= 1.0.0"
  required_providers {
    ibm = {
      source  = "IBM-Cloud/ibm"
      version = "1.31.0"
    }
  }
}

##################################################################
## Create RG
##################################################################

resource "ibm_resource_group" "test_resource_group" {
  name     = "${var.prefix}-rg"
  quota_id = null
}

##################################################################
## Create Secrets Manager instance
##################################################################

resource "ibm_resource_instance" "secrets_manager" {
  name              = "${var.prefix}-sm-instance"
  service           = "secrets-manager"
  plan              = "lite"
  location          = "us-south"
  resource_group_id = ibm_resource_group.test_resource_group.id
  timeouts {
    create = "20m" // Extending provisioning time to 20 minutes
  }
}

Debug Output

sm-trace-log.txt

Panic Output

Expected Behavior

Correct response should be returned so user knows the root cause as to why provisioning failed.

Actual Behavior

Unexpected HTTP status code 503 (503 Service Unavailable) with resp code: null

Steps to Reproduce

1. terraform apply

Important Factoids

References

• #0000

[kavya498]

@ocofaigh ,
https://github.com/IBM-Cloud/terraform-provider-ibm/blob/master/ibm/resource_ibm_resource_instance.go#L459
We just return the error what API gives us..

We use resource-controller APIs and SDK and return same response that API gives us

Is UI calling same APIs?
we are not sure if they are doing any additional calls..

[ocofaigh]

@kavya498 The RC api does return correct response:

{"error_code":"RC-ServiceBrokerErrorResponse","message":"[400, Bad Request] This account has reached the limit of 1 Secrets Manager instances. If you're unable to access the existing Secrets Manager instance from your IBM Cloud resource list, contact your administrator to verify your permissions.","details":"{\"error\":\"Bad Request\",\"description\":\"This account has reached the limit of 1 Secrets Manager instances. If you're unable to access the existing Secrets Manager instance from your IBM Cloud resource list, contact your administrator to verify your permissions.\"}","status_code":503,"transaction_id":"bss-9c791c6edf026582"}

I think maybe you guys are not parsing it correctly?

[kavya498]

@ocofaigh , After investigation, We found that SDK is not returning proper error response..
We ll take this to SDK team.. Thanks for identifying this issue..

[ocofaigh]

The is not specific to secrets-manager, it is any resource being provisioned by RC api. Updating the title.

[hkantare]

@ocofaigh We identified the issue when we intialize the clients with enable retries from SDK we are not able to get back the complete response but the same without retries enabled the we are able to fetch comlete respose..We are in touch with SDK genenrator team @dustin Popp and @phil_adams

[ocofaigh]

@hkantare Thanks for the update. I saw this response today when trying to delete a resource group. Does it also have the same problem?

Error Deleting resource group: Unexpected HTTP status code 500 (500 Internal Server Error) with responce code  null

[hkantare]

@ocofaigh Yes it should be the same problem not returning complete response..Once we receive the complete response then we get more details why actually resource group deletion is failing..

Based on response we can analayze future steps...

The SDK team fixed the issue of response and mostly it will be part of Novemeber releases