Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix Resources Failing with TF_SCHEMA_PANIC_ON_ERROR #13312

Closed
12 tasks done
bflad opened this issue May 13, 2020 · 2 comments · Fixed by #13395
Closed
12 tasks done

Fix Resources Failing with TF_SCHEMA_PANIC_ON_ERROR #13312

bflad opened this issue May 13, 2020 · 2 comments · Fixed by #13395
Assignees
@appilon
Labels
bug Addresses a defect in current functionality.
Milestone
v2.64.0

Comments

@bflad
Copy link
Contributor

bflad commented May 13, 2020 •
edited by appilon
Loading

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

The acceptance testing framework has supported the TF_SCHEMA_PANIC_ON_ERROR environment variable as a methodology for causing a panic instead of (usually unchecked) errors with d.Set(). In Terraform Plugin SDK version 2, this behavior will be enabled (in some form, potentially not a panic) by default so provider developers can more easily discover these trickier coding issues. Static analysis and the potential to add typed d.Get()/d.Set() receiver methods cannot prevent this type of issue everywhere before it reaches testing.

These errors indicate places where Terraform cannot properly set Computed attribute values in the state or properly perform drift detection for arguments.

Affected Resource(s)

Currently failing:

  • aws_lb_cookie_stickiness_policy: cookie_expiration_period: '' expected type 'int', got unconvertible type 'string'
  • aws_lb_ssl_negotiation_policy: Invalid address to set: []string{"attributes"}
  • aws_lightsail_instance: ram_size: '' expected type 'int', got unconvertible type 'string'
  • aws_load_balancer_listener_policy: load_balancer_port: '' expected type 'int', got unconvertible type 'string'
  • aws_load_balancer_backend_server_policy: instance_port: '' expected type 'int', got unconvertible type 'string'
  • aws_load_balancer_policy: load_balancer_port: '' expected type 'int', got unconvertible type 'string'
  • aws_opsworks_application: environment.0.secure: '' expected type 'bool', got unconvertible type 'string'
  • aws_redshift_security_group: Invalid address to set: []string{"skip_final_snapshot"} (on import)
  • aws_sns_topic: lambda_success_feedback_sample_rate: '' expected type 'int', got unconvertible type 'string'
  • aws_ssm_activation: expired: '' expected type 'string', got unconvertible type 'bool'
  • aws_ssm_document: Invalid address to set: []string{"document_version"}
  • several others

References
@bflad bflad added the bug Addresses a defect in current functionality. label May 13, 2020
@bflad bflad mentioned this issue May 13, 2020
Closed
18 tasks
This was referenced May 15, 2020
Closed
Merged
Merged
Merged
Merged
Merged
Merged
Merged
Merged
Merged
Merged
Merged
Merged
Merged
Merged
Merged
Closed
@appilon appilon added this to the v2.64.0 milestone May 27, 2020
@ghost
Copy link

ghost commented May 29, 2020

This has been released in version 2.64.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

bflad added a commit that referenced this issue Jun 23, 2020
@bflad
resource/aws_autoscaling_group: Prevent unexpected differences in `ta…
6df2b91 
…gs` for Terraform 0.11 and earlier with boolean `propagate_at_launch` values

Reference: #13312
Reference: #13360
Reference: #13469
Reference: #13549
Reference: #13635

One of the major improvements internally with Terraform 0.12's type system is that providers and the core logic agree on a resource schema and its normalization during configuration parsing so the state and difference handling is more consistent. In Terraform 0.11 and earlier, a quirky behavior with raw boolean values is that they could be converted to string "0" and "1" values based on various factors such as passing through functions. With recent changes to the `aws_autoscaling_group` resource to fix its handling of the `tags` attribute (to prevent `d.Set()` errors and testing panics in the upcoming Terraform Plugin SDK 2.0.0), this behavior was highlighted as unexpected perpetual differences. This cannot be caught in the 0.12 shimmed acceptance testing framework and therefore is manually tested by the below configuration and reproduction steps.

**NOTE:** The `tags` argument will be deprecated and removed in a future major version of the Terraform AWS Provider since it was designed as a workaround for Terraform 0.11 and earlier configurations. There are very few, if any, other Terraform AWS Provider attributes represented as a set of maps. In Terraform 0.12 and later, the `tag` argument more appropriately represents Auto Scaling Group resource tags as configuration blocks with strongly typed nested arguments and can be dynamically built via the `dynamic` configuration language feature. That deprecation is out of scope for this changeset though and likely will not occur until after Terraform AWS Provider version 3.0.0.

Given the following Terraform 0.11 compatible configuration with raw boolean values:

```hcl
terraform {
  required_version = "0.11.14"
}

provider "aws" {
  region  = "us-east-2"
  version = "2.67.0"
}

locals {
  test = [
    {
      key                 = "localkey1"
      value               = "localvalue1"
      propagate_at_launch = true
    },
    {
      key                 = "localkey2"
      value               = "localvalue2"
      propagate_at_launch = true
    },
  ]
}

data "aws_ami" "test" {
  most_recent = true
  owners      = ["amazon"]

  filter {
    name   = "name"
    values = ["amzn-ami-hvm-*-x86_64-gp2"]
  }
}

data "aws_availability_zones" "available" {
  state = "available"

  filter {
    name   = "opt-in-status"
    values = ["opt-in-not-required"]
  }
}

resource "aws_launch_template" "test" {
  name_prefix   = "test"
  image_id      = "${data.aws_ami.test.id}"
  instance_type = "t2.micro"
}

resource "aws_autoscaling_group" "test" {
  availability_zones = ["${data.aws_availability_zones.available.names[0]}"]
  desired_capacity   = 0
  max_size           = 0
  min_size           = 0
  tags               = ["${concat(list(map("key", "resourcekey1", "value", "resourcevalue1", "propagate_at_launch", true)), local.test)}"]

  launch_template {
    id      = "${aws_launch_template.test.id}"
    version = "${aws_launch_template.test.default_version}"
  }
}

output "tags" {
  value = "${aws_autoscaling_group.test.tags}"
}

```

Applying twice has a perpetual difference:

```console
$ terraform0.11.14 init

Initializing provider plugins...
- Checking for available provider plugins on https://releases.hashicorp.com...
- Downloading plugin for provider "aws" (2.67.0)...

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

$ terraform0.11.14 apply
data.aws_ami.test: Refreshing state...
data.aws_availability_zones.available: Refreshing state...

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  + aws_autoscaling_group.test
      id:                                  <computed>
      arn:                                 <computed>
      availability_zones.#:                "1"
      availability_zones.4293815384:       "us-east-2a"
      default_cooldown:                    <computed>
      desired_capacity:                    "0"
      force_delete:                        "false"
      health_check_grace_period:           "300"
      health_check_type:                   <computed>
      launch_template.#:                   "1"
      launch_template.0.id:                "${aws_launch_template.test.id}"
      launch_template.0.name:              <computed>
      launch_template.0.version:           "${aws_launch_template.test.default_version}"
      load_balancers.#:                    <computed>
      max_size:                            "0"
      metrics_granularity:                 "1Minute"
      min_size:                            "0"
      name:                                <computed>
      protect_from_scale_in:               "false"
      service_linked_role_arn:             <computed>
      tags.#:                              "3"
      tags.2825254636.%:                   "3"
      tags.2825254636.key:                 "localkey1"
      tags.2825254636.propagate_at_launch: "1"
      tags.2825254636.value:               "localvalue1"
      tags.2888644505.%:                   "3"
      tags.2888644505.key:                 "resourcekey1"
      tags.2888644505.propagate_at_launch: "1"
      tags.2888644505.value:               "resourcevalue1"
      tags.3299348900.%:                   "3"
      tags.3299348900.key:                 "localkey2"
      tags.3299348900.propagate_at_launch: "1"
      tags.3299348900.value:               "localvalue2"
      target_group_arns.#:                 <computed>
      vpc_zone_identifier.#:               <computed>
      wait_for_capacity_timeout:           "10m"

  + aws_launch_template.test
      id:                                  <computed>
      arn:                                 <computed>
      default_version:                     <computed>
      image_id:                            "ami-083ebc5a49573896a"
      instance_type:                       "t2.micro"
      latest_version:                      "0"
      metadata_options.#:                  <computed>
      name:                                <computed>
      name_prefix:                         "test"

Plan: 2 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

aws_launch_template.test: Creating...
  arn:                "" => "<computed>"
  default_version:    "" => "<computed>"
  image_id:           "" => "ami-083ebc5a49573896a"
  instance_type:      "" => "t2.micro"
  latest_version:     "" => "<computed>"
  metadata_options.#: "" => "<computed>"
  name:               "" => "<computed>"
  name_prefix:        "" => "test"
aws_launch_template.test: Creation complete after 0s (ID: lt-0d8e6f4f94a683778)
aws_autoscaling_group.test: Creating...
  arn:                                 "" => "<computed>"
  availability_zones.#:                "" => "1"
  availability_zones.4293815384:       "" => "us-east-2a"
  default_cooldown:                    "" => "<computed>"
  desired_capacity:                    "" => "0"
  force_delete:                        "" => "false"
  health_check_grace_period:           "" => "300"
  health_check_type:                   "" => "<computed>"
  launch_template.#:                   "" => "1"
  launch_template.0.id:                "" => "lt-0d8e6f4f94a683778"
  launch_template.0.name:              "" => "<computed>"
  launch_template.0.version:           "" => "1"
  load_balancers.#:                    "" => "<computed>"
  max_size:                            "" => "0"
  metrics_granularity:                 "" => "1Minute"
  min_size:                            "" => "0"
  name:                                "" => "<computed>"
  protect_from_scale_in:               "" => "false"
  service_linked_role_arn:             "" => "<computed>"
  tags.#:                              "" => "3"
  tags.2825254636.%:                   "" => "3"
  tags.2825254636.key:                 "" => "localkey1"
  tags.2825254636.propagate_at_launch: "" => "1"
  tags.2825254636.value:               "" => "localvalue1"
  tags.2888644505.%:                   "" => "3"
  tags.2888644505.key:                 "" => "resourcekey1"
  tags.2888644505.propagate_at_launch: "" => "1"
  tags.2888644505.value:               "" => "resourcevalue1"
  tags.3299348900.%:                   "" => "3"
  tags.3299348900.key:                 "" => "localkey2"
  tags.3299348900.propagate_at_launch: "" => "1"
  tags.3299348900.value:               "" => "localvalue2"
  target_group_arns.#:                 "" => "<computed>"
  vpc_zone_identifier.#:               "" => "<computed>"
  wait_for_capacity_timeout:           "" => "10m"
aws_autoscaling_group.test: Creation complete after 2s (ID: tf-asg-20200623194008467200000003)

Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

Outputs:

tags = [
    {
        key = localkey2,
        propagate_at_launch = 1,
        value = localvalue2
    },
    {
        key = localkey1,
        propagate_at_launch = 1,
        value = localvalue1
    },
    {
        key = resourcekey1,
        propagate_at_launch = 1,
        value = resourcevalue1
    }
]

$ terraform0.11.14 apply
data.aws_ami.test: Refreshing state...
data.aws_availability_zones.available: Refreshing state...
aws_launch_template.test: Refreshing state... (ID: lt-0d8e6f4f94a683778)
aws_autoscaling_group.test: Refreshing state... (ID: tf-asg-20200623194008467200000003)

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  ~ aws_autoscaling_group.test
      tags.2131493069.%:                   "3" => "0"
      tags.2131493069.key:                 "localkey2" => ""
      tags.2131493069.propagate_at_launch: "true" => ""
      tags.2131493069.value:               "localvalue2" => ""
      tags.2825254636.%:                   "0" => "3"
      tags.2825254636.key:                 "" => "localkey1"
      tags.2825254636.propagate_at_launch: "" => "1"
      tags.2825254636.value:               "" => "localvalue1"
      tags.2888644505.%:                   "0" => "3"
      tags.2888644505.key:                 "" => "resourcekey1"
      tags.2888644505.propagate_at_launch: "" => "1"
      tags.2888644505.value:               "" => "resourcevalue1"
      tags.3299348900.%:                   "0" => "3"
      tags.3299348900.key:                 "" => "localkey2"
      tags.3299348900.propagate_at_launch: "" => "1"
      tags.3299348900.value:               "" => "localvalue2"
      tags.3593646732.%:                   "3" => "0"
      tags.3593646732.key:                 "localkey1" => ""
      tags.3593646732.propagate_at_launch: "true" => ""
      tags.3593646732.value:               "localvalue1" => ""
      tags.3707627218.%:                   "3" => "0"
      tags.3707627218.key:                 "resourcekey1" => ""
      tags.3707627218.propagate_at_launch: "true" => ""
      tags.3707627218.value:               "resourcevalue1" => ""

Plan: 0 to add, 1 to change, 0 to destroy.
```

Building this version of the provider, the difference is no longer present:

```console
$ cp ~/go/bin/terraform-provider-aws .terraform/plugins/darwin_amd64/terraform-provider-aws_v2.67.0_x4; terraform0.11.14 init; terraform0.11.14 apply

Initializing provider plugins...

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
data.aws_availability_zones.available: Refreshing state...
data.aws_ami.test: Refreshing state...
aws_launch_template.test: Refreshing state... (ID: lt-0d8e6f4f94a683778)
aws_autoscaling_group.test: Refreshing state... (ID: tf-asg-20200623194008467200000003)

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Outputs:

tags = [
    {
        key = localkey1,
        propagate_at_launch = 1,
        value = localvalue1
    },
    {
        key = resourcekey1,
        propagate_at_launch = 1,
        value = resourcevalue1
    },
    {
        key = localkey2,
        propagate_at_launch = 1,
        value = localvalue2
    }
]
```

Output from acceptance testing (Terraform 0.12 compatible):

```
--- PASS: TestAccAWSAutoScalingGroup_ALB_TargetGroups (176.05s)
--- PASS: TestAccAWSAutoScalingGroup_ALB_TargetGroups_ELBCapacity (342.40s)
--- PASS: TestAccAWSAutoScalingGroup_autoGeneratedName (89.91s)
--- PASS: TestAccAWSAutoScalingGroup_basic (259.25s)
--- PASS: TestAccAWSAutoScalingGroup_classicVpcZoneIdentifier (93.45s)
--- PASS: TestAccAWSAutoScalingGroup_emptyAvailabilityZones (92.72s)
--- PASS: TestAccAWSAutoScalingGroup_enablingMetrics (174.98s)
--- PASS: TestAccAWSAutoScalingGroup_initialLifecycleHook (271.36s)
--- PASS: TestAccAWSAutoScalingGroup_launchTemplate (62.35s)
--- PASS: TestAccAWSAutoScalingGroup_LaunchTemplate_IAMInstanceProfile (61.69s)
--- PASS: TestAccAWSAutoScalingGroup_launchTemplate_update (143.24s)
--- PASS: TestAccAWSAutoScalingGroup_launchTempPartitionNum (77.30s)
--- PASS: TestAccAWSAutoScalingGroup_LoadBalancers (695.77s)
--- PASS: TestAccAWSAutoScalingGroup_MaxInstanceLifetime (80.89s)
--- PASS: TestAccAWSAutoScalingGroup_MixedInstancesPolicy (108.93s)
--- PASS: TestAccAWSAutoScalingGroup_MixedInstancesPolicy_InstancesDistribution_OnDemandAllocationStrategy (49.98s)
--- PASS: TestAccAWSAutoScalingGroup_MixedInstancesPolicy_InstancesDistribution_OnDemandBaseCapacity (82.14s)
--- PASS: TestAccAWSAutoScalingGroup_MixedInstancesPolicy_InstancesDistribution_OnDemandPercentageAboveBaseCapacity (96.59s)
--- PASS: TestAccAWSAutoScalingGroup_MixedInstancesPolicy_InstancesDistribution_SpotAllocationStrategy (83.60s)
--- PASS: TestAccAWSAutoScalingGroup_MixedInstancesPolicy_InstancesDistribution_SpotInstancePools (90.32s)
--- PASS: TestAccAWSAutoScalingGroup_MixedInstancesPolicy_InstancesDistribution_SpotMaxPrice (112.90s)
--- PASS: TestAccAWSAutoScalingGroup_MixedInstancesPolicy_InstancesDistribution_UpdateToZeroOnDemandBaseCapacity (73.62s)
--- PASS: TestAccAWSAutoScalingGroup_MixedInstancesPolicy_LaunchTemplate_LaunchTemplateSpecification_LaunchTemplateName (86.69s)
--- PASS: TestAccAWSAutoScalingGroup_MixedInstancesPolicy_LaunchTemplate_LaunchTemplateSpecification_Version (85.84s)
--- PASS: TestAccAWSAutoScalingGroup_MixedInstancesPolicy_LaunchTemplate_Override_InstanceType (85.29s)
--- PASS: TestAccAWSAutoScalingGroup_MixedInstancesPolicy_LaunchTemplate_Override_WeightedCapacity (205.06s)
--- PASS: TestAccAWSAutoScalingGroup_namePrefix (45.70s)
--- PASS: TestAccAWSAutoScalingGroup_serviceLinkedRoleARN (83.67s)
--- PASS: TestAccAWSAutoScalingGroup_suspendingProcesses (267.51s)
--- PASS: TestAccAWSAutoScalingGroup_tags (299.92s)
--- PASS: TestAccAWSAutoScalingGroup_TargetGroupArns (249.92s)
--- PASS: TestAccAWSAutoScalingGroup_terminationPolicies (151.45s)
--- PASS: TestAccAWSAutoScalingGroup_VpcUpdates (236.40s)
--- PASS: TestAccAWSAutoScalingGroup_WithLoadBalancer (389.18s)
--- PASS: TestAccAWSAutoScalingGroup_WithLoadBalancer_ToTargetGroup (384.27s)
--- PASS: TestAccAWSAutoScalingGroup_withMetrics (96.85s)
--- PASS: TestAccAWSAutoScalingGroup_withPlacementGroup (200.69s)
```
@bflad bflad mentioned this issue Jun 23, 2020
Merged
@ghost
Copy link

ghost commented Jun 27, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

@ghost ghost locked and limited conversation to collaborators Jun 27, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@appilon appilon

Labels
bug Addresses a defect in current functionality.
Projects
None yet
Milestone
v2.64.0
2 participants
@bflad @appilon