filter_current_dashboard of newrelic_one_dashboard is not handled properly

[dvishniakov]

Please include the following with your bug report

• Your New Relic provider configuration (sensitive details redacted)
• A list of affected resources and/or data sources
• The configuration of the resources and/or data sources related to the bug report (i.e. from the list mentioned above)
• Description of the current behavior (the bug)
• Description of the expected behavior
• Any related log output

Terraform Version

1.0.9

Affected Resource(s)

Please list the resources as a list, for example:

• newrelic_one_dashboard
• one_dashboard_raw (according to Bug/Regression: Inability to filter by facet with one_dashboard_raw #1413 )

Terraform Configuration

Any basic config of newrelic_one_dashboard with widget_table and filter_current_dashboard=true

Actual Behavior

Terraform plan and apply operations show changes every time:

      ~ page {
...
          ~ widget_table {
              ~ filter_current_dashboard = false -> true

Expected Behavior

no changes

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

1. terraform apply or terraform plan (after apply)

Debug Output

Panic Output

Important Factoids

References

[rdhar]

I was referring to this anomalous behaviour in #1413 so I really appreciate you raising this issue to address directly. Thanks.

[dvishniakov]

Thanks @rdhar , I've added one_dashboard_raw to the affected resources list with a hope that all of them could be fixed at once.

[achupryn]

The issue is still present in v2.33.0

$terraform -version
Terraform v1.0.11
on linux_amd64
+ provider registry.terraform.io/newrelic/newrelic v2.33.0

[joshua-paypay]

Issue

Hi, this is still not working correctly in 2.34.1.

$ terraform -version
Terraform v1.1.2
on darwin_amd64
+ provider registry.terraform.io/newrelic/newrelic v2.34.1

I had some issue setting up dashboards originated from a json file copy from an old dashboard.
The original dashboard had linked_entity_guids = [] to the old dashboard.

widget_bar {
  # omitting details
  linked_entity_guids = ["old-dashboard-guid"]
}

This works fine, however, it is pointing to the old dashboard.

So, I've changed the config to

widget_bar {
  # omitting details
  filter_current_dashboard = true
}

The dashboard still points to the old one.

Experiments

I did some experiments

1. I created a dashboard like my real dashboard (business details omitted). Then both work fine. Widget 1 can filter the test dashboard.

resource "newrelic_one_dashboard" "test-dashboard" {

  page {
    widget_bar {
      name = "widget 1"
      filter_current_dashboard =  true
    }

    widget_bar {
       name = "widget 2"
      linked_entity_guids = ["old-dashboard-guid"]
    }
  }
}

2. I removed the guids from widget 2. Then Widget 2 had no filters. This is also fine.

resource "newrelic_one_dashboard" "test-dashboard" {

  page {
    widget_bar {
      name = "widget 1"
      filter_current_dashboard =  true
    }

    widget_bar {
       name = "widget 2"
      linked_entity_guids = []
    }
  }
}

3. I add filter_current_dashbaord = true to widget 2. However no filter is generated.

resource "newrelic_one_dashboard" "test-dashboard" {

  page {
    widget_bar {
      name = "widget 1"
      filter_current_dashboard =  true
    }

    widget_bar {
       name = "widget 2"
      filter_current_dashboard =  true
    }
  }
}

[MrColeC]

We are having the same issue.

More data:

• I found on a fresh Dashboard creation, the feature works and the filtering is possible
• However on subsequent Terraform applies the filtering can be removed from the widgets and it is not possible to add it back via TF without tainting the entire dashboards state and executing a complete destroy/create
• Adding the toggle in the UI (filter current dashboard UI toggle in New Relic's UI for that dashboard) does not reflect or show up as a state change for this item
• If you delete the dashboard page in the UI, and run a TF apply it will not restore the page to a working order (with filtering enabled) so it is possible to use this feature for initial dashboard + pages creation, but not for any subsequent updates to that dashboard or generation of fresh/new dashboard pages

[zeffron]

@joshua-paypay and @MrColeC your issue sounds like a separate issue from the one the original post was about. Could you maybe open a second issue to track that?
@dvishniakov Could you maybe adjust the title of this issue to be more specific to the plan showing changes every time?

[MrColeC]

The experience I have is the same as what is reported:

Actual Behavior
Terraform plan and apply operations show changes every time:

      ~ page {
...
          ~ widget_table {
              ~ filter_current_dashboard = false -> true

Even though the state of the dashboard is unchanged, and terraform applied successfully, subsequent back-to-back TF runs express that the value needs to change from false to true (but never do). Thus, the feature does not work; "filter_current_dashboard of newrelic_one_dashboard is not handled properly".

[zeffron]

@MrColeC I may have incorrectly mixed your and @joshua-paypay's experiences together. My apologies. I saw that you said you were having "the same issue" immediately after their comment, which seems to have different expected behavior from the original post.

[MrColeC]

Sorry I am being confusing. Thank you for discussing this.
I do not expect those changes to be there.

1. We try to use this feature (filter_current_dashboard)
2. It creates the dashboard, and it works as expected in the UI (yay!)
3. I immediately run the same TF apply (I do not change any TF) and it says that it has not made the state updates, and filter_current_dashboard is false and needs to be true (not good)

The TF state and the New Relic UI are not in sync.
Further, updates to a widget that is working will "break" the filtering and it cannot be added back using this same TF, which is confusing.

[zeffron]

@MrColeC Gotcha, thank you for clarifying. It sounds like you have two (possibly related) issues then.

1. The one this one was opened for, which is the inaccurate indication of needing to (re)apply filter_current_dashboard
2. Updates to the widget can cause descync between reality and Terraform management.

Can you elaborate on what updates you've seen this desync with? Are these changes made to the widget via the UI? Or changes made with Terraform?

[joshua-paypay]

@zeffron #1604 new issue created

[MrColeC]

Context:
I write TF to create a dashboard, using the following Terraform.

terraform {
  required_providers {
    newrelic = {
      source  = "newrelic/newrelic"
      version = ">= 2.35.1"
    }
  }
}

provider "newrelic" {
  account_id = "<redacted>"
  region     = "US"
}

resource "newrelic_one_dashboard" "default" {
  name        = "Ticket 1494"
  description = "Managed via Terraform"

  page {
    name        = "Simple Page"
    description = "Simple Page"

    # Pie charts
    widget_pie {
      title                    = "Filter Example"
      row                      = 1
      column                   = 1
      width                    = 5
      height                   = 5
      linked_entity_guids      = null
      filter_current_dashboard = true
      nrql_query {
        query = "FROM K8sClusterSample select uniqueCount(hostname) FACET clusterName limit max since 5 minutes ago"
      }
    }

    widget_pie {
      title                    = "Target Example"
      row                      = 1
      column                   = 6
      width                    = 7
      height                   = 5
      linked_entity_guids      = null
      filter_current_dashboard = false
      nrql_query {
        query = "FROM K8sClusterSample select uniqueCount(hostname) FACET instanceType limit max since 5 minutes ago"
      }
    }
  }
}

Results:
After an apply, run a TF plan and it says it needs to still set the filter to true. Which is not true.
Further, if you use TF to edit that pie chart, the filter function will "drop" and you cannot use TF to add it back.
Additionally, if you toggle on that it is filtering (via the web UI) the TF state refresh will not detect it.

Result details:

terraform plan
newrelic_one_dashboard.default: Refreshing state... [id=<redacted>]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # newrelic_one_dashboard.default will be updated in-place
  ~ resource "newrelic_one_dashboard" "default" {
        id          = "<redacted>"
        name        = "Ticket 1494"
        # (5 unchanged attributes hidden)

      ~ page {
            name        = "Simple Page"
            # (2 unchanged attributes hidden)

          ~ widget_pie {
              ~ filter_current_dashboard = false -> true
                id                       = "<redacted>"
                # (6 unchanged attributes hidden)

                # (1 unchanged block hidden)
            }
            # (1 unchanged block hidden)
        }
    }

Plan: 0 to add, 1 to change, 0 to destroy.

[MrColeC]

Does this help @zeffron ?
Sorry for the delay in response.

[zeffron]

@MrColeC Yes, thank you!

[rdhar]

While re-trying #1413, I came across a li'l tidbit that might help address this issue (i.e., unnecessary noise in the execution logs) with an interim solution: leveraging ignore_changes from Terraform's lifecycle meta-argument docs.

resource "newrelic_one_dashboard" "the_dashboard" {
  lifecycle {
    ignore_changes = [
      page[0].widget_bar["filter_current_dashboard"],
      page[0].widget_pie["filter_current_dashboard"],
      page[0].widget_table["filter_current_dashboard"],
    ]
  }
  name = "The Dashboard"
  …

This will suppress redundant noise from filter_current_dashboard on subsequent updates. Almost goes without saying, the ideal solution here would still be for the provider to properly register the value of the boolean, but this isn't too shabby a workaround.

[MrColeC]

Support for Lifecycle only exists on Resources, not Dynamic blocks. As the nature of a Dashboard can be complex at times and most have multiple pages and widgets, Dynamics are generally used.

hashicorp/terraform#24188

I agree this helps the example, but for us it is not going to help in production.

[rdhar]

Hey @dvishniakov & @MrColeC, could you retry your use-cases?

Following the release of v2.38.0 , I've found it addressed some issues relating to dashboard filters.

[MrColeC]

@rdhar I do see in some limited use cases, or base case deployments with no follow on updates, it is functional. However, at scale and on updates, I still see issues. I think the issue persists.

I looked over the PR's merged in that made up v2.38.0 and I didn't see any changes I would think are associated with this issue.

[mbazhlekova]

Hey folks 👋 we're trying something new with this issue. We released a fix in a beta version - v2.41.0-beta.1. We would really appreciate it if you could grab this release and let us know if it fixes the issues you're seeing for filter_current_dashboard. Thanks!

[MrColeC]

Thank you @mbazhlekova !! Testing it now

[MrColeC]

@mbazhlekova our testing and our use cases all seem to work with this. This is a full solution and we see no side effects. Nice!

[zeffron]

Can confirm the beta seems to work for our use case as well.

[MrColeC]

I am now trying an update on a dashboard and getting

...
module.newrelic.newrelic_one_dashboard.default: Modifying... 
│ Error: err: filter_current_dashboard can't be set if linked_entity_guids is configured
...

I am looking into if that is an issue with how we use this, or upstream.

[MrColeC]

Based upon what I can read on our side, I do not think we are causing this error condition.

I do believe that we are tripping old safety code that was added late last year to prevent someone from attempting to configure this same feature from two different ways.

Suspected source:
https://github.com/newrelic/terraform-provider-newrelic/pull/1460/files#diff-c52fab50bb8226f3765aaff0b68ef743b70e6d4d28b05c367bcf947e4abaa439R385

[mbazhlekova]

@MrColeC We intentionally don't allow linked_entity_guids and filter_current_dashboard to both be set. Since they both set the linked_entity_guids field, it can lead to conflicts. Are you seeing this error in another use case?

[MrColeC]

We are not setting it, we are just using filter_current_dashboard, which is setting in the background linked_entity_guids but now that filter_current_dashboard is being correctly persisted in state, on an update it trips the logic.

To reproduce:

terraform {
  required_providers {
    newrelic = {
      source  = "newrelic/newrelic"
      version = "=2.41.0-beta.1"
    }
  }
}

provider "newrelic" {
  account_id = "<id>"
  region     = "US"
}

resource "newrelic_one_dashboard" "default" {
  name        = "Ticket 1494"
  description = "Managed via Terraform"

  page {
    name        = "Simple Page"
    description = "Simple Page"

    # Pie charts
    widget_pie {
      title                    = "Filter Example"
      row                      = 1
      column                   = 1
      width                    = 5
      height                   = 5
      linked_entity_guids      = null
      filter_current_dashboard = true
      nrql_query {
        query = "FROM K8sClusterSample select uniqueCount(hostname) FACET clusterName limit max since 5 minutes ago"
      }
    }

    widget_pie {
      title                    = "Target Example"
      row                      = 1
      column                   = 6
      width                    = 7
      height                   = 5
      linked_entity_guids      = null
      filter_current_dashboard = false
      nrql_query {
        query = "FROM K8sClusterSample select uniqueCount(hostname) FACET instanceType limit max since 5 minutes ago"
      }
    }
  }
}

1. Terraform init
2. Terraform apply
3. Terraform apply (2nd time, no changes! NICE! This is where we got stuck before)
4. Now change the NRQL query in a pie widget from since 5 minutes ago to since 10 minutes ago to ask for an update
5. Terraform apply (error)

[zeffron]

@MrColeC Depending on Terraform version, I think it can tell the difference between something explicitly set to null, and something not set. Does it still reproduce if you delete the lines with the unused one (filter_current_dashboard when it's set to false, and linked_entity_guids when filter is true)?

[MrColeC]

The test code I am using to reproduce this I posted above. When trying what you said (good idea!) it still did not work.

Error: err: filter_current_dashboard can't be set if linked_entity_guids is configured

I believe the reason why is that filter_current_dashboard sets linked_entity_guids in the background. Thus, the implementation forces the failure of its own safety check when attempting to update something it can now properly create.

[zeffron]

Ah, yes, I missed the change to the NRQL statement initially. I am able to reproduce the same failure.

[mbazhlekova]

@MrColeC I see, thanks for the clarification! Looks like we need to refine the logic a bit to get it to work properly.

[mbazhlekova]

@MrColeC @zeffron We rolled out another release - v2.41.0-beta.2 with a fix for the linked_entity_guids error. Please let us know if this resolves the issues you're seeing.

[zeffron]

v2.41.0-beta.2 does seem to have resolved the issue for me.

[MrColeC]

I agree with @zeffron
@mbazhlekova this has fixed it for me and I have not found any edge cases. Thank you for fixing this!