Import of google_cloud_asset_folder_feed forces replacement despite matching config #9461

mchesler · 2021-06-29T16:05:58Z

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
If you are interested in working on this issue or have submitted a pull request, please leave a comment.
If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Terraform Version

Terraform version 0.12.31
Google Provider version 3.74.0

Affected Resource(s)

google_cloud_asset_folder_feed

Terraform Configuration Files

resource "google_cloud_asset_folder_feed" "my_feed" {
  billing_project = "my-project"
  folder          = "folders/1234567890"
  feed_id         = "my-feed"
  content_type    = "RESOURCE"
  asset_types     = ["compute.googleapis.com/Instance"]

  feed_output_config {
    pubsub_destination {
      topic = "projects/my-project/topics/instance-events"
    }
  }
}

Expected Behavior

After import, terraform plan should show no changes to apply

Actual Behavior

After import, terraform plan shows output like below:

  # google_cloud_asset_folder_feed.my_feed must be replaced
-/+ resource "google_cloud_asset_folder_feed" "my_feed" {
      - asset_names   = [] -> null
        asset_types   = [
          "compute.googleapis.com/Instance",
        ]
      + billing_project = "my-project" # forces replacement
        content_type    = "RESOURCE"
      + feed_id         = "my-feed" # forces replacement
      + folder          = "folders/1234567890" # forces replacement
      + folder_id       = (known after apply)
      ~ id              = "folders/1234567890/feeds/my-feed" -> (known after apply)
      ~ name            = "folders/1234567890/feeds/my-feed" -> (known after apply)

        feed_output_config {
          pubsub_destination {
            topic = "projects/my-project/topics/instance-events"
          }
        }

      - timeouts {}
    }

Steps to Reproduce

terraform import google_cloud_asset_folder_feed.my-feed folders/1234567890/feeds/my-feed
terraform plan

Important Factoids

None

References

None

b/374161733

The text was updated successfully, but these errors were encountered:

edwardmedia · 2021-06-30T13:52:56Z

@mchesler can you see if below records are in the state?

      + billing_project = "my-project" # forces replacement
      + feed_id         = "my-feed" # forces replacement
      + folder          = "folders/1234567890" # forces replacement

mchesler · 2021-06-30T18:58:59Z

@edwardmedia no, I see this in the state:

{
  "mode": "managed",
  "type": "google_cloud_asset_folder_feed",
  "name": "my_feed",
  "provider": "provider.google",
  "instances": [
    {
      "schema_version": 0,
      "attributes": [
        "asset_names": [
        ],
        "asset_types": [
        ]
      ],
      "billing_project": null,
      "condition": [
      ],
      "content_type": "RESOURCE",
      "feed_id": null,
      "feed_output_config": [
        {
          "pubsub_destination": [
            {
              "topic": "projects/my-project/topics/instance-events"
            }
          ]
        }
      ],
      "folder": null,
      "folder_id": "",
      "id": "folders/1234567890/feeds/my-feed",
      "name": "folders/1234567890/feeds/my-feed",
      "timeouts": {
          "create": null,
          "delete": null,
          "update": null,
        }
      },
      "private": "abcdef1234567890abcdef1234567890abcdef1234567890"
    }
  ]
}

edwardmedia · 2021-07-01T16:16:04Z

@mchesler below fields are null or empty in the state. That is why. Can you check the debug log to see if they were included in the response from api? If not, you may need to re-apply the config to sync the config. After that, let's check to see if you still see the difference.

      "billing_project": null,
      "feed_id": null,
      "folder": null,
      "folder_id": "",

mchesler · 2021-07-01T21:32:34Z

@edwardmedia Looks like those fields were not included in the API response:

2021-07-01T17:30:09.317-0400 [INFO]  plugin.terraform-provider-google_v3.74.0_x5: 2021/07/01 17:30:09 [DEBUG] Google API Request Details:
---[ REQUEST ]---------------------------------------
GET /v1/folders/1234567890/feeds/my-feed?alt=json HTTP/1.1
Host: cloudasset.googleapis.com
User-Agent: Terraform/0.12.31 (+https://www.terraform.io) Terraform-Plugin-SDK/2.5.0 terraform-provider-google/3.74.0
Content-Type: application/json
Accept-Encoding: gzip


-----------------------------------------------------: timestamp=2021-07-01T17:30:09.317-0400
2021-07-01T17:30:09.867-0400 [INFO]  plugin.terraform-provider-google_v3.74.0_x5: 2021/07/01 17:30:09 [DEBUG] Google API Response Details:
---[ RESPONSE ]--------------------------------------
HTTP/2.0 200 OK
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: private
Content-Type: application/json; charset=UTF-8
Date: Thu, 01 Jul 2021 21:30:10 GMT
Server: ESF
Vary: Origin
Vary: X-Origin
Vary: Referer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 0

{
  "name": "folders/1234567890/feeds/my-feed",
  "assetTypes": [
"compute.googleapis.com/Instance"
  ],
  "contentType": "RESOURCE",
  "feedOutputConfig": {
    "pubsubDestination": {
      "topic": "projects/my-project/topics/instance-events"
    }
  }
}

-----------------------------------------------------: timestamp=2021-07-01T17:30:09.867-0400

The issue with re-applying is as I originally stated above - terraform is going to force replacement of the resource, which I do not want to happen.

edwardmedia · 2021-07-01T22:14:05Z

@mchesler exactly. The provider does not have control over what api sends back. Did you create the resource via Terraform? If not, did you provided the data for these fields? I understood you didn't want to re-apply, but by doing so we may be able to see if these fields are captured by API. Make sense?

Looks like those fields were not included in the API response

edwardmedia · 2021-07-19T19:41:46Z

@mchesler is this still an issue?

mchesler · 2021-07-19T19:50:17Z

@edwardmedia yes, still an issue. It seems as though the values get stored in terraform state when terraform creates the resource, so it's not constantly trying to destroy and recreate. While I can recreate at will for testing, I'm not able to destroy and recreate the resource in production, so I'm somewhat stuck.

edwardmedia · 2021-07-19T22:30:39Z

@mchesler was the state created via imported or when you created the resource ? Did something happen after the resource initially created?

I am trying to understand (the below) how the Terraform forces replacement despite matching config?

Import of google_cloud_asset_folder_feed forces replacement despite matching config

mchesler · 2021-07-20T13:40:01Z

@edwardmedia the state appears to be created correctly when I create the resource via terraform apply, but not via import. Here's the debug output from the creation via terraform apply:

google_cloud_asset_folder_feed.my_feed: Creating...
2021/07/20 09:22:48 [DEBUG] google_cloud_asset_folder_feed.my_feed: applying the planned Create change
2021-07-20T09:22:48.012-0400 [INFO]  plugin.terraform-provider-google_v3.74.0_x5: 2021/07/20 09:22:48 [DEBUG] Creating new FolderFeed: map[string]interface {}{"feed":map[string]interface {}{"assetTypes":[]interface {}{"compute.googleapis.com/Instance"}, "contentType":"RESOURCE", "feedOutputConfig":map[string]interface {}{"pubsubDestination":map[string]interface {}{"topic":"projects/my-project/topics/instance-events"}}}}: timestamp=2021-07-20T09:22:48.012-0400
2021-07-20T09:22:48.012-0400 [INFO]  plugin.terraform-provider-google_v3.74.0_x5: 2021/07/20 09:22:48 [DEBUG] Waiting for state to become: [success]: timestamp=2021-07-20T09:22:48.012-0400
2021-07-20T09:22:48.012-0400 [INFO]  plugin.terraform-provider-google_v3.74.0_x5: 2021/07/20 09:22:48 [DEBUG] Retry Transport: starting RoundTrip retry loop: timestamp=2021-07-20T09:22:48.012-0400
2021-07-20T09:22:48.012-0400 [INFO]  plugin.terraform-provider-google_v3.74.0_x5: 2021/07/20 09:22:48 [DEBUG] Retry Transport: request attempt 0: timestamp=2021-07-20T09:22:48.012-0400
2021-07-20T09:22:48.013-0400 [INFO]  plugin.terraform-provider-google_v3.74.0_x5: 2021/07/20 09:22:48 [DEBUG] Google API Request Details:
---[ REQUEST ]---------------------------------------
POST /v1/folders/1234567890/feeds?alt=json&feedId=my-feed HTTP/1.1
Host: cloudasset.googleapis.com
User-Agent: Terraform/0.12.31 (+https://www.terraform.io) Terraform-Plugin-SDK/2.5.0 terraform-provider-google/3.74.0
Content-Length: 180
Content-Type: application/json
X-Goog-User-Project: my-project
Accept-Encoding: gzip

{
 "feed": {
  "assetTypes": [
   "compute.googleapis.com/Instance"
  ],
  "contentType": "RESOURCE",
  "feedOutputConfig": {
   "pubsubDestination": {
    "topic": "projects/my-project/topics/instance-events"
   }
  }
 }
}

-----------------------------------------------------: timestamp=2021-07-20T09:22:48.013-0400
2021-07-20T09:22:48.907-0400 [INFO]  plugin.terraform-provider-google_v3.74.0_x5: 2021/07/20 09:22:48 [DEBUG] Google API Response Details:
---[ RESPONSE ]--------------------------------------
HTTP/2.0 200 OK
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: private
Content-Type: application/json; charset=UTF-8
Date: Tue, 20 Jul 2021 13:22:48 GMT
Server: ESF
Vary: Origin
Vary: X-Origin
Vary: Referer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 0

{
  "name": "folders/1234567890/feeds/my-feed",
  "assetTypes": [
"compute.googleapis.com/Instance"
  ],
  "contentType": "RESOURCE",
  "feedOutputConfig": {
    "pubsubDestination": {
      "topic": "projects/my-project/topics/instance-events"
    }
  }
}

-----------------------------------------------------: timestamp=2021-07-20T09:22:48.907-0400
2021-07-20T09:22:48.908-0400 [INFO]  plugin.terraform-provider-google_v3.74.0_x5: 2021/07/20 09:22:48 [DEBUG] Retry Transport: Stopping retries, last request was successful: timestamp=2021-07-20T09:22:48.907-0400
2021-07-20T09:22:48.908-0400 [INFO]  plugin.terraform-provider-google_v3.74.0_x5: 2021/07/20 09:22:48 [DEBUG] Retry Transport: Returning after 1 attempts: timestamp=2021-07-20T09:22:48.907-0400
2021-07-20T09:22:48.908-0400 [INFO]  plugin.terraform-provider-google_v3.74.0_x5: 2021/07/20 09:22:48 [DEBUG] Finished creating FolderFeed "folders/1234567890/feeds/my-feed": map[string]interface {}{"assetTypes":[]interface {}{"compute.googleapis.com/Instance"}, "contentType":"RESOURCE", "feedOutputConfig":map[string]interface {}{"pubsubDestination":map[string]interface {}{"topic":"projects/my-project/topics/instance-events"}}, "name":"folders/1234567890/feeds/my-feed"}: timestamp=2021-07-20T09:22:48.908-0400
2021-07-20T09:22:48.908-0400 [INFO]  plugin.terraform-provider-google_v3.74.0_x5: 2021/07/20 09:22:48 [DEBUG] Waiting for state to become: [success]: timestamp=2021-07-20T09:22:48.908-0400
2021-07-20T09:22:48.908-0400 [INFO]  plugin.terraform-provider-google_v3.74.0_x5: 2021/07/20 09:22:48 [DEBUG] Retry Transport: starting RoundTrip retry loop: timestamp=2021-07-20T09:22:48.908-0400
2021-07-20T09:22:48.908-0400 [INFO]  plugin.terraform-provider-google_v3.74.0_x5: 2021/07/20 09:22:48 [DEBUG] Retry Transport: request attempt 0: timestamp=2021-07-20T09:22:48.908-0400
2021-07-20T09:22:48.908-0400 [INFO]  plugin.terraform-provider-google_v3.74.0_x5: 2021/07/20 09:22:48 [DEBUG] Google API Request Details:
---[ REQUEST ]---------------------------------------
GET /v1/folders/1234567890/feeds/my-feed?alt=json HTTP/1.1
Host: cloudasset.googleapis.com
User-Agent: Terraform/0.12.31 (+https://www.terraform.io) Terraform-Plugin-SDK/2.5.0 terraform-provider-google/3.74.0
Content-Type: application/json
Accept-Encoding: gzip


-----------------------------------------------------: timestamp=2021-07-20T09:22:48.908-0400
2021-07-20T09:22:49.122-0400 [INFO]  plugin.terraform-provider-google_v3.74.0_x5: 2021/07/20 09:22:49 [DEBUG] Google API Response Details:
---[ RESPONSE ]--------------------------------------
HTTP/2.0 200 OK
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: private
Content-Type: application/json; charset=UTF-8
Date: Tue, 20 Jul 2021 13:22:49 GMT
Server: ESF
Vary: Origin
Vary: X-Origin
Vary: Referer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 0

{
  "name": "folders/1234567890/feeds/my-feed",
  "assetTypes": [
"compute.googleapis.com/Instance"
  ],
  "contentType": "RESOURCE",
  "feedOutputConfig": {
    "pubsubDestination": {
      "topic": "projects/my-project/topics/instance-events"
    }
  }
}

-----------------------------------------------------: timestamp=2021-07-20T09:22:49.122-0400
2021-07-20T09:22:49.122-0400 [INFO]  plugin.terraform-provider-google_v3.74.0_x5: 2021/07/20 09:22:49 [DEBUG] Retry Transport: Stopping retries, last request was successful: timestamp=2021-07-20T09:22:49.122-0400
2021-07-20T09:22:49.122-0400 [INFO]  plugin.terraform-provider-google_v3.74.0_x5: 2021/07/20 09:22:49 [DEBUG] Retry Transport: Returning after 1 attempts: timestamp=2021-07-20T09:22:49.122-0400
google_cloud_asset_folder_feed.my_feed: Creation complete after 1s [id=folders/1234567890/feeds/my-feed]

And the resulting state shows the correct information:

$ terraform state show google_cloud_asset_folder_feed.my_feed                                                                                                                                                    [21-07-20 9:31:25]
# google_cloud_asset_folder_feed.my_feed:
resource "google_cloud_asset_folder_feed" "my_feed" {
    asset_names     = []
    asset_types     = [
        "compute.googleapis.com/Instance",
    ]
    billing_project = "my-project"
    content_type    = "RESOURCE"
    feed_id         = "my-feed"
    folder          = "folders/1234567890"
    id              = "folders/1234567890/feeds/my-feed"
    name            = "folders/1234567890/feeds/my-feed"

    feed_output_config {
        pubsub_destination {
            topic = "projects/my-project/topics/instance-events"
        }
    }
}

However, if I remove and reimport the resource, the state looks like this:

$ terraform state show google_cloud_asset_folder_feed.my_feed                                                                                                                                                    [21-07-20 9:34:25]
# google_cloud_asset_folder_feed.my_feed:
resource "google_cloud_asset_folder_feed" "my_feed" {
    asset_names  = []
    asset_types  = [
        "compute.googleapis.com/Instance",
    ]
    content_type = "RESOURCE"
    id           = "folders/1234567890/feeds/my-feed"
    name         = "folders/1234567890/feeds/my-feed"

    feed_output_config {
        pubsub_destination {
            topic = "projects/my-project/topics/instance-events"
        }
    }

    timeouts {}
}

To recap, the API responses on create vs import look identical, but resource creation results in correct/complete terraform state, while resource import does not and forces resource replacement on the next apply.

edwardmedia · 2021-07-20T18:14:32Z

@mchesler I think I understood your issue now. After import, you still need to construct the resource config, google_cloud_asset_folder_feed manually, by adding those fields with the data obtained from the import. There is no automatic process that can handle that in the provider. Do I address your issue?

mchesler · 2021-07-20T18:25:30Z

@edwardmedia I think that covers the issue. Stated slightly differently - after importing a google_cloud_assets_folder_feed resource, the real state of the resource is not accurately reflected in terraform state.

edwardmedia · 2021-07-22T19:13:47Z

I see. We can populate folder and feed_id for the import but not the billing_project as it is not directly available. Add label of persistent-bug

[upstream:5320b7984e43c4f7ff843058ea029551327e4a06] Signed-off-by: Modular Magician <magic-modules@google.com>

roaks3 · 2024-10-25T18:48:54Z

I've confirmed this is still an issue. Import is effectively broken for this resource, in that it requires a subsequent recreation to resolve (or hand-editing the state).

Recommendation is to:

Update the import id to include billing_project, since it can't be fetch from the API. So it might look something like:

identity:
  - billingProject

Update the custom import to parse the name, and explicitly set folder, folder_id, feed_id, and billing_project. None of these are populated automatically during read. It might help to change self_link to folders/{{folder_id}}/feeds/{{feed_id}}.

Note that a similar fix might be needed for the other *Feed resources.

mchesler added the bug label Jun 29, 2021

edwardmedia self-assigned this Jun 29, 2021

edwardmedia added the waiting-response label Jun 30, 2021

github-actions bot removed the waiting-response label Jun 30, 2021

edwardmedia added the waiting-response label Jun 30, 2021

github-actions bot removed the waiting-response label Jun 30, 2021

edwardmedia added the waiting-response label Jul 1, 2021

github-actions bot removed the waiting-response label Jul 1, 2021

edwardmedia added the waiting-response label Jul 1, 2021

github-actions bot removed the waiting-response label Jul 1, 2021

edwardmedia added the waiting-response label Jul 1, 2021

github-actions bot removed the waiting-response label Jul 19, 2021

edwardmedia added the waiting-response label Jul 19, 2021

github-actions bot removed the waiting-response label Jul 19, 2021

edwardmedia added the waiting-response label Jul 19, 2021

github-actions bot removed the waiting-response label Jul 19, 2021

edwardmedia added the waiting-response label Jul 19, 2021

github-actions bot removed the waiting-response label Jul 20, 2021

edwardmedia added the waiting-response label Jul 20, 2021

github-actions bot removed the waiting-response label Jul 20, 2021

edwardmedia added the waiting-response label Jul 20, 2021

github-actions bot removed the waiting-response label Jul 20, 2021

edwardmedia added the persistent-bug Hard to diagnose or long lived bugs for which resolutions are more like feature work than bug work label Jul 22, 2021

edwardmedia removed their assignment Jul 22, 2021

rileykarson added size/s and removed bug labels Jul 22, 2021

rileykarson added this to the Goals milestone Jul 26, 2021

github-actions bot added forward/review In review; remove label to forward service/cloudasset labels Aug 17, 2023

modular-magician added a commit that referenced this issue Nov 13, 2023

Update doc links for lifecycle meta argument (#9461) (#16526)

473488f

[upstream:5320b7984e43c4f7ff843058ea029551327e4a06] Signed-off-by: Modular Magician <magic-modules@google.com>

melinath removed the forward/review In review; remove label to forward label Oct 17, 2024

modular-magician added the forward/linked label Oct 17, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Import of google_cloud_asset_folder_feed forces replacement despite matching config #9461

Import of google_cloud_asset_folder_feed forces replacement despite matching config #9461

mchesler commented Jun 29, 2021 •

edited by modular-magician

Loading

edwardmedia commented Jun 30, 2021

mchesler commented Jun 30, 2021

edwardmedia commented Jul 1, 2021

mchesler commented Jul 1, 2021 •

edited

Loading

edwardmedia commented Jul 1, 2021

edwardmedia commented Jul 19, 2021

mchesler commented Jul 19, 2021

edwardmedia commented Jul 19, 2021 •

edited

Loading

mchesler commented Jul 20, 2021

edwardmedia commented Jul 20, 2021

mchesler commented Jul 20, 2021

edwardmedia commented Jul 22, 2021

roaks3 commented Oct 25, 2024

Import of google_cloud_asset_folder_feed forces replacement despite matching config #9461

Import of google_cloud_asset_folder_feed forces replacement despite matching config #9461

Comments

mchesler commented Jun 29, 2021 • edited by modular-magician Loading

Community Note

Terraform Version

Affected Resource(s)

Terraform Configuration Files

Expected Behavior

Actual Behavior

Steps to Reproduce

Important Factoids

References

edwardmedia commented Jun 30, 2021

mchesler commented Jun 30, 2021

edwardmedia commented Jul 1, 2021

mchesler commented Jul 1, 2021 • edited Loading

edwardmedia commented Jul 1, 2021

edwardmedia commented Jul 19, 2021

mchesler commented Jul 19, 2021

edwardmedia commented Jul 19, 2021 • edited Loading

mchesler commented Jul 20, 2021

edwardmedia commented Jul 20, 2021

mchesler commented Jul 20, 2021

edwardmedia commented Jul 22, 2021

roaks3 commented Oct 25, 2024

mchesler commented Jun 29, 2021 •

edited by modular-magician

Loading

mchesler commented Jul 1, 2021 •

edited

Loading

edwardmedia commented Jul 19, 2021 •

edited

Loading