data source aws_ec2_transit_gateway_route_table doesn't feed data properly in plan phase to resource "aws_ec2_transit_gateway_route" #11554

tbugfinder · 2020-01-10T17:28:23Z

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

Terraform v0.12.19

provider.aws v2.44.0
provider.random v2.2.1

Affected Resource(s)

aws_ec2_transit_gateway_route
data source: aws_ec2_transit_gateway_route_table

Terraform Configuration Files

provider "random" {
}

resource "random_id" "id" {
  byte_length = 4
}

locals {
  id = random_id.id.hex
}

locals {
  name = "tgw-test"
}

resource "aws_ec2_transit_gateway" "tgw-private" {
  tags = {
    Name = "${local.name}-private-${local.id}"
  }
}

resource "aws_ec2_transit_gateway" "tgw-public" {
  tags = {
    Name = "${local.name}-public-${local.id}"
  }
}

###############################################
# Maintain private Route Table of TGWA Routes #
###############################################
data "aws_ec2_transit_gateway_route_table" "tgwrtb" {
  filter {
    name   = "transit-gateway-id"
    values = [aws_ec2_transit_gateway.tgw-private.id]
  }
}

resource "aws_ec2_transit_gateway_route" "add_private_route_1" {
  destination_cidr_block         = "192.168.1.0/27"
  transit_gateway_attachment_id  = "tgw-1111111111"
  transit_gateway_route_table_id = data.aws_ec2_transit_gateway_route_table.tgwrtb.id
}

Debug Output

Panic Output

Expected Behavior

Terraform should start creating the resources and querying the data source once needed before creating the resource.

Actual Behavior

terraform analyzes the code and cannot resolve the data source properly so it fails.

$ terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.


------------------------------------------------------------------------

Error: "transit_gateway_route_table_id": required field is not set

  on main.tf line 38, in resource "aws_ec2_transit_gateway_route" "add_private_route_1":
  38: resource "aws_ec2_transit_gateway_route" "add_private_route_1" {

Steps to Reproduce

terraform plan

Important Factoids

References

#0000

The text was updated successfully, but these errors were encountered:

tbugfinder · 2020-01-10T22:56:21Z

This code was developed with tf 0.11.14 and migrated to tf 0.12.18.
There wasn't an error before.

tbugfinder · 2020-01-12T21:42:35Z

Workaround: add null data source between tgwrtb data source and resource add_private_route_1.

dthvt · 2020-01-16T18:51:28Z

Is there a reason you can't use

transit_gateway_route_table_id = aws_ec2_transit_gateway.tgw-private.association_default_route_table_id

tbugfinder · 2020-01-24T16:51:02Z

Well, at the stage the question rather was why would it fail with tf0.12?

tbugfinder · 2020-01-24T16:52:50Z

provider.aws v2.46.0 also fails.

dthvt · 2020-01-24T17:54:47Z

You may be seeing an issue w/ AWS eventual consistency. In other words, the TGW creation may come back as successful, but the describe-transit-gateway API may not be reporting its existence right away.

tbugfinder · 2020-01-24T18:17:51Z

The sample code fails already in planning stage. So this isn't dependent on eventual consistency. It doesn't honor input of data.aws_ec2_transit_gateway_route_table.tgwrtb.id .

dthvt · 2020-01-24T18:27:03Z

I would reiterate my suggestion of using the alternate code I provided. Does that work?

ewbankkit · 2020-05-06T14:38:22Z

add retries to data source(s) aws_ec2_transit_gateway_vpc_attachments #10769

tbugfinder · 2020-11-27T23:12:58Z

tf0.13 seems to be even stricter :-)
I gave above line a try which makes tf0.13 happy.

transit_gateway_route_table_id = aws_ec2_transit_gateway.tgw-private.association_default_route_table_id

tbugfinder · 2020-12-03T17:07:18Z

After doing some additional testing I have to get back to this.
In case the TGW is not created within the same module I have to use the tgwid and a data source to get the IDs.

I looks more like an issue with the data source than the resource.

tbugfinder · 2020-12-03T19:33:27Z

pasting TF_LOG output

2020-12-03T20:26:22.256+0100 [DEBUG] plugin.terraform-provider-aws_v3.19.0_x5: </DescribeAccountAttributesResponse>
2020/12/03 20:26:22 [DEBUG] Resource instance state not found for node "aws_ec2_transit_gateway.tgw", instance aws_ec2_transit_gateway.tgw
2020/12/03 20:26:22 [WARN] ReferenceTransformer: reference not found: "local.name"
2020/12/03 20:26:22 [WARN] ReferenceTransformer: reference not found: "local.id"
2020/12/03 20:26:22 [DEBUG] ReferenceTransformer: "aws_ec2_transit_gateway.tgw" references: []
2020/12/03 20:26:22 [DEBUG] Resource instance state not found for node "module.mymodule.aws_customer_gateway.cgw", instance module.mymodule.aws_customer_gateway.cgw
2020/12/03 20:26:22 [WARN] ReferenceTransformer: reference not found: "var.ip_address"
2020/12/03 20:26:22 [WARN] ReferenceTransformer: reference not found: "local.merged_tags"
2020/12/03 20:26:22 [WARN] ReferenceTransformer: reference not found: "var.vpn_type"
2020/12/03 20:26:22 [WARN] ReferenceTransformer: reference not found: "var.bgp_asn"
2020/12/03 20:26:22 [DEBUG] ReferenceTransformer: "module.mymodule.aws_customer_gateway.cgw" references: []
2020/12/03 20:26:22 [WARN] Provider "registry.terraform.io/hashicorp/aws" produced an invalid plan for module.mymodule.aws_customer_gateway.cgw, but we are tolerating it because it is
 using the legacy plugin SDK.
    The following problems may be the cause of any confusing errors from downstream operations:
      - .tags: planned value cty.UnknownVal(cty.Map(cty.String)) does not match config value cty.MapVal(map[string]cty.Value{"Name":cty.UnknownVal(cty.String)})
2020/12/03 20:26:22 [WARN] Provider "registry.terraform.io/hashicorp/aws" produced an invalid plan for aws_ec2_transit_gateway.tgw, but we are tolerating it because it is using the legacy plugin S
DK.
    The following problems may be the cause of any confusing errors from downstream operations:
      - .tags: planned value cty.UnknownVal(cty.Map(cty.String)) does not match config value cty.MapVal(map[string]cty.Value{"Name":cty.UnknownVal(cty.String)})
      - .vpn_ecmp_support: planned value cty.StringVal("enable") does not match config value cty.NullVal(cty.String)
      - .dns_support: planned value cty.StringVal("enable") does not match config value cty.NullVal(cty.String)
      - .amazon_side_asn: planned value cty.NumberIntVal(64512) does not match config value cty.NullVal(cty.Number)
      - .auto_accept_shared_attachments: planned value cty.StringVal("disable") does not match config value cty.NullVal(cty.String)
      - .default_route_table_association: planned value cty.StringVal("enable") does not match config value cty.NullVal(cty.String)
      - .default_route_table_propagation: planned value cty.StringVal("enable") does not match config value cty.NullVal(cty.String)
2020/12/03 20:26:22 [DEBUG] Resource instance state not found for node "module.mymodule.aws_vpn_connection.vpn", instance module.mymodule.aws_vpn_connection.vpn
2020/12/03 20:26:22 [WARN] ReferenceTransformer: reference not found: "var.static_routes_only"
2020/12/03 20:26:22 [WARN] ReferenceTransformer: reference not found: "var.vpn_type"
2020/12/03 20:26:22 [WARN] ReferenceTransformer: reference not found: "local.merged_tags"
2020/12/03 20:26:22 [WARN] ReferenceTransformer: reference not found: "var.transit_gateway_id"
2020/12/03 20:26:22 [DEBUG] ReferenceTransformer: "module.mymodule.aws_vpn_connection.vpn" references: []
2020/12/03 20:26:22 [DEBUG] Resource instance state not found for node "module.mymodule.data.aws_ec2_transit_gateway_route_table.tgwrtb", instance module.mymodule.data.aw
s_ec2_transit_gateway_route_table.tgwrtb
2020/12/03 20:26:22 [WARN] ReferenceTransformer: reference not found: "var.transit_gateway_id"
2020/12/03 20:26:22 [DEBUG] ReferenceTransformer: "module.mymodule.data.aws_ec2_transit_gateway_route_table.tgwrtb" references: []
2020/12/03 20:26:22 [WARN] Provider "registry.terraform.io/hashicorp/aws" produced an invalid plan for module.mymodule.aws_vpn_connection.vpn, but we are tolerating it because it is u
sing the legacy plugin SDK.
    The following problems may be the cause of any confusing errors from downstream operations:
      - .tags: planned value cty.UnknownVal(cty.Map(cty.String)) does not match config value cty.MapVal(map[string]cty.Value{"Name":cty.UnknownVal(cty.String)})
2020/12/03 20:26:22 [DEBUG] Resource instance state not found for node "module.mymodule.data.external.tgwlookup", instance module.mymodule.data.external.tgwlookup
2020/12/03 20:26:22 [WARN] ReferenceTransformer: reference not found: "var.transit_gateway_id"
2020/12/03 20:26:22 [WARN] ReferenceTransformer: reference not found: "path.module"

tbugfinder · 2020-12-03T19:34:36Z

During the plan phase the data source is obviously empty.
``
2020/12/03 20:26:22 [DEBUG] ReferenceTransformer: "module.mymodule.data.aws_ec2_transit_gateway_route_table.tgwrtb" references: []

dthvt · 2020-12-04T15:55:12Z

I believe that's expected behavior of Terraform. To quote https://www.terraform.io/docs/configuration/data-sources.html

If the query constraint arguments for a data resource refer only to constant values or values that are already known, the data resource will be read and its state updated during Terraform's "refresh" phase, which runs prior to creating a plan. This ensures that the retrieved data is available for use during planning and so Terraform's plan will show the actual values obtained.

Query constraint arguments may refer to values that cannot be determined until after configuration is applied, such as the id of a managed resource that has not been created yet. In this case, reading from the data source is deferred until the apply phase, and any references to the results of the data resource elsewhere in configuration will themselves be unknown until after the configuration has been applied.

tbugfinder · 2020-12-04T19:22:36Z

Actually that's exactly not my understanding:
... may refer to values that cannot be determined until after configuration is applied ... reading from the data source is deferred until the apply phase ...

tbugfinder · 2020-12-05T20:59:09Z

So, I'm making the value itself dependant on the filter item within the data source and apply can be executed successfully (0.12.29, 0.13.5, 0.14.0)

transit_gateway_route_table_id = length(aws_ec2_transit_gateway.tgw-private.id) > 4 ? data.aws_ec2_transit_gateway_route_table.tgwrtb.id : "no-tgw

apparentlymart · 2020-12-08T20:20:51Z

Hi all,

I was working through this problem with @tbugfinder in a community forum topic and so I wanted to just report back and share what we learned there.

The short version (please see the forum topic for the long version!) is that the data source aws_ec2_transit_gateway_route_table seems to have an incorrect schema which marks its id attribute as being optional but not computed, which in Terraform's schema model means that it's a configuration argument only and not something that will be populated as part of reporting the data source result.

{
  "type": "string",
  "description_kind": "plain",
  "optional": true
}

Because of that, Terraform Core sees that it isn't set in the configuration and uses null as the value for it, which is the correct way for Terraform to represent an argument that wasn't set. The downstream reference to it then correctly fails validation, because null isn't a valid value for transit_gateway_route_table_id.

Marking this id attribute as "computed": true in the schema would make this work as expected I think, because then Terraform would know that the provider intends to provide its own value for that attribute when it isn't set in the configuration, and thus generate a plan with that attribute set to (known after apply) rather than null.

dthvt · 2020-12-08T20:28:23Z

That code appears out of date. Have you updated your code to the latest AWS provider? The current code shows:

			"id": {
				Type:     schema.TypeString,
				Optional: true,
			},

tbugfinder · 2020-12-08T21:22:32Z

If I swap id with arn the plan (I didn't apply) finishes successfully. Indeed the difference is the computed: true vs. optional.

Looking at provider 3.20 code base:
https://github.com/hashicorp/terraform-provider-aws/blob/v3.20.0/aws/data_source_aws_ec2_transit_gateway_route_table.go#L15-L44

			"arn": {
				Type:     schema.TypeString,
				Computed: true,
			},
....
....
			"id": {
				Type:     schema.TypeString,
				Optional: true,
			},

"Unfortunately" or obviously apply fails if arn is used :-D
I'll try to patch and build a provider locally.

bflad · 2020-12-09T03:20:32Z

From some quick searching, some affected data sources:

aws_customer_gateway
aws_ec2_transit_gateway_peering_attachment
aws_ec2_transit_gateway_route_table
aws_ec2_transit_gateway_vpc_attachment
aws_ec2_transit_gateway
aws_guardduty_detector

Adding Computed: true to these id attributes certainly seems like a good update for these. Then again, it would probably be best if the Terraform Plugin SDK either failed schema validation when this attribute behavior is missing or automatically added it, since the id attribute is implicitly required to have a value (hashicorp/terraform-plugin-sdk#541). I will followup upstream there as well.

tbugfinder · 2020-12-09T12:01:56Z

After getting the provider built with Computed: true within id, I can confirm that the plan finishes successfully - I'm investigating the apply phase.

tbugfinder · 2020-12-09T13:13:12Z

apply phase finished successfully.

bflad · 2020-12-09T17:44:45Z

The fix for the noted data sources above has been merged and will release with version 3.21.0 of the Terraform AWS Provider, likely tomorrow. Thank you @apparentlymart and @tbugfinder. 👍

ghost · 2020-12-11T17:12:28Z

This has been released in version 3.21.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

ghost · 2021-01-09T17:10:01Z

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

ghost added the service/ec2 Issues and PRs that pertain to the ec2 service. label Jan 10, 2020

github-actions bot added the needs-triage Waiting for first response or review from a maintainer. label Jan 10, 2020

tbugfinder changed the title ~~resource "aws_ec2_transit_gateway_route" doesn't properly depend on data sources~~ data source aws_ec2_transit_gateway_route_table doesn't feed data properly in plan phase to resource "aws_ec2_transit_gateway_route" Dec 3, 2020

bflad added bug Addresses a defect in current functionality. and removed needs-triage Waiting for first response or review from a maintainer. labels Dec 9, 2020

bflad mentioned this issue Dec 9, 2020

Prevent Invalid id Attribute Schema (Missing Computed) hashicorp/terraform-plugin-sdk#654

Open

tbugfinder mentioned this issue Dec 9, 2020

adding Computed :true to id of few data sources #16667

Merged

bflad added this to the v3.21.0 milestone Dec 9, 2020

bflad closed this as completed in #16667 Dec 9, 2020

ghost locked as resolved and limited conversation to collaborators Jan 9, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

data source aws_ec2_transit_gateway_route_table doesn't feed data properly in plan phase to resource "aws_ec2_transit_gateway_route" #11554

data source aws_ec2_transit_gateway_route_table doesn't feed data properly in plan phase to resource "aws_ec2_transit_gateway_route" #11554

tbugfinder commented Jan 10, 2020 •

edited

Loading

tbugfinder commented Jan 10, 2020

tbugfinder commented Jan 12, 2020

dthvt commented Jan 16, 2020

tbugfinder commented Jan 24, 2020

tbugfinder commented Jan 24, 2020

dthvt commented Jan 24, 2020

tbugfinder commented Jan 24, 2020

dthvt commented Jan 24, 2020

ewbankkit commented May 6, 2020

tbugfinder commented Nov 27, 2020 •

edited

Loading

tbugfinder commented Dec 3, 2020

tbugfinder commented Dec 3, 2020

tbugfinder commented Dec 3, 2020 •

edited

Loading

dthvt commented Dec 4, 2020

tbugfinder commented Dec 4, 2020 •

edited

Loading

tbugfinder commented Dec 5, 2020

apparentlymart commented Dec 8, 2020

dthvt commented Dec 8, 2020

tbugfinder commented Dec 8, 2020 •

edited

Loading

bflad commented Dec 9, 2020

tbugfinder commented Dec 9, 2020 •

edited

Loading

tbugfinder commented Dec 9, 2020

bflad commented Dec 9, 2020

ghost commented Dec 11, 2020

ghost commented Jan 9, 2021

data source aws_ec2_transit_gateway_route_table doesn't feed data properly in plan phase to resource "aws_ec2_transit_gateway_route" #11554

data source aws_ec2_transit_gateway_route_table doesn't feed data properly in plan phase to resource "aws_ec2_transit_gateway_route" #11554

Comments

tbugfinder commented Jan 10, 2020 • edited Loading

Community Note

Terraform Version

Affected Resource(s)

Terraform Configuration Files

Debug Output

Panic Output

Expected Behavior

Actual Behavior

Steps to Reproduce

Important Factoids

References

tbugfinder commented Jan 10, 2020

tbugfinder commented Jan 12, 2020

dthvt commented Jan 16, 2020

tbugfinder commented Jan 24, 2020

tbugfinder commented Jan 24, 2020

dthvt commented Jan 24, 2020

tbugfinder commented Jan 24, 2020

dthvt commented Jan 24, 2020

ewbankkit commented May 6, 2020

tbugfinder commented Nov 27, 2020 • edited Loading

tbugfinder commented Dec 3, 2020

tbugfinder commented Dec 3, 2020

tbugfinder commented Dec 3, 2020 • edited Loading

dthvt commented Dec 4, 2020

tbugfinder commented Dec 4, 2020 • edited Loading

tbugfinder commented Dec 5, 2020

apparentlymart commented Dec 8, 2020

dthvt commented Dec 8, 2020

tbugfinder commented Dec 8, 2020 • edited Loading

bflad commented Dec 9, 2020

tbugfinder commented Dec 9, 2020 • edited Loading

tbugfinder commented Dec 9, 2020

bflad commented Dec 9, 2020

ghost commented Dec 11, 2020

ghost commented Jan 9, 2021

tbugfinder commented Jan 10, 2020 •

edited

Loading

tbugfinder commented Nov 27, 2020 •

edited

Loading

tbugfinder commented Dec 3, 2020 •

edited

Loading

tbugfinder commented Dec 4, 2020 •

edited

Loading

tbugfinder commented Dec 8, 2020 •

edited

Loading

tbugfinder commented Dec 9, 2020 •

edited

Loading