provider initialization and dependencies have race condition when used with eauch other

[byahia]

Hi there,

Context

The provider initialization and dependencies with any other resource have race condition between each other when used togerther, no way to use depends_on to workaround it

Terraform Version

Terraform v0.12.25

Provider version

• provider.aws v2.64.0
• provider.helm v1.2.2
• provider.kubernetes v1.11.2
• provider.local v1.4.0
• provider.null v2.1.2
• provider.postgresql v1.6.0
• provider.random v2.2.1

Affected Resource(s)

Provider initialization

Debug Output

provider "postgresql" {
  host           = module.my_db.this_db_instance_address
  port            =  module.my_db.this_db_instance_port
  username  = "myusername"
  password   = var.my_database_password

}

Error: Incorrect attribute value type

  on ../../../../modules/jfrog_artifactory/rds.tf line 65, in provider "postgresql":
  65:   port            = module.jfrog_db.this_db_instance_port
    |----------------
    | module.jfrog_db.this_db_instance_port is ""

Inappropriate value for attribute "port": a number is required.

Expected Behavior

Wait for the provider initialization if the dependent resources isn't created yet.

Actual Behavior

The provider initialize before waiting for the dependencies to complete creating.

[dploeger]

Workaround: Use the expected_version parameter to specify the PostgreSQL version you'll be deploying later.

[Aluxima]

Specifying expected_version allows using the provider to deploy a postgresql resource once, then still getting Error: could not start transaction: dial tcp :0: connect: connection refused the following times.

Also using terraform-aws-rds output in the provider configuration.

Versions:
Terraform v0.13.2

• provider registry.terraform.io/hashicorp/aws v3.7.0
• provider registry.terraform.io/terraform-providers/postgresql v1.7.1

[tcondeixa]

Specifying expected_version allows using the provider to deploy a postgresql resource once, then still getting Error: could not start transaction: dial tcp :0: connect: connection refused the following times.

Also using terraform-aws-rds output in the provider configuration.

Versions:
Terraform v0.13.2

• provider registry.terraform.io/hashicorp/aws v3.7.0
• provider registry.terraform.io/terraform-providers/postgresql v1.7.1

I'm having the same problem after changing to terraform 0.13, maybe it's related with the changes on providers with new terraform version. The host of the provider is depending on the output from my RDS module with the respective DB endpoint, but I'm receiving Error: dial tcp :5432: connect: connection refused. I did a simple test on defining directly the endpoint string in the host instead of using the output of the module and it worked, but I need it to be dynamic.

[dnrce]

https://www.terraform.io/docs/configuration/providers.html#provider-configuration-1 says:

You can use expressions in the values of these configuration arguments, but can only reference values that are known before the configuration is applied. This means you can safely reference input variables, but not attributes exported by resources (with an exception for resource arguments that are specified directly in the configuration).

Does this mean this is impossible to do? Does the overall design of requiring host in the provider initialization need to be revisited?

[tcondeixa]

https://www.terraform.io/docs/configuration/providers.html#provider-configuration-1 says:

You can use expressions in the values of these configuration arguments, but can only reference values that are known before the configuration is applied. This means you can safely reference input variables, but not attributes exported by resources (with an exception for resource arguments that are specified directly in the configuration).

Does this mean this is impossible to do? Does the overall design of requiring host in the provider initialization need to be revisited?

I have been setting the host as the output from the RDS module with the trick of setting the expected_version to avoid problems during the init phase. This has been working fine for several months, but when I tried to upgrade to terraform 0.13 it is failing after the first apply.

[tcondeixa]

Is there any new regarding this topic? do you plan to look into this problem? I'm blocked to update to terraform 0.13 at the moment in all states using postgres provider, so it would be great to have some feedback. thanks a lot

[tcondeixa]

I found a workaround for this problem, maybe it could be useful for others. One of the main changes of terraform 0.13 is that the host of the provider cannot depend on an output from a module anymore but it can depend on the attributes exported by resources. So I just created a data terraform type to get the address from the DB in RDS. I just forced the data resource to depend on some output from the module that does the DB creation to be sure it is executed after. You have here an example for AWS RDS.

provider "postgresql" {
  host            = data.aws_db_instance.database.address
  database    = var.db_name
  ....
}

data "aws_db_instance" "database" {
  db_instance_identifier = var.instance_name

  depends_on = [module.db.depends]
}