Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Plugin crash with "set item just set doesn't exist" #277

Closed
josecsotomorales opened this issue May 15, 2023 · 8 comments · Fixed by #349
Closed

Plugin crash with "set item just set doesn't exist" #277

josecsotomorales opened this issue May 15, 2023 · 8 comments · Fixed by #349
Labels
bug Something isn't working cannot-reproduce Unable to reproduce issue.

Comments

@josecsotomorales
Copy link

Terraform Version, ArgoCD Provider Version and ArgoCD Version

Terraform version: 1.4.5
ArgoCD provider version: 5.3.0
ArgoCD version: 2.7.1

Affected Resource(s)

  • argocd_application

Panic Output

╷
│ Error: Request cancelled
│ 
│   with argocd_application.helm,
│   on argocd_add_app.tf line 5, in resource "argocd_application" "helm":
│    5: resource "argocd_application" "helm" {
│ 
│ The plugin.(*GRPCProvider).ReadResource request was cancelled.
╵

Stack trace from the terraform-provider-argocd_v5.3.0 plugin:

panic: set item just set doesn't exist

goroutine 99 [running]:
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*MapFieldWriter).setSet(0xc001029758, {0xc0010fb2d0, 0x7, 0x7}, {0x288a0a0, 0xc001029860}, 0xc001030f00)
	github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.1/helper/schema/field_writer_map.go:327 +0x959
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*MapFieldWriter).set(0xc001029758, {0xc0010fb2d0, 0x7, 0x7}, {0x288a0a0, 0xc001029860})
	github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.1/helper/schema/field_writer_map.go:107 +0x14c
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*MapFieldWriter).setObject(0xc0004eab40?, {0xc0004eab40?, 0x6, 0x6}, {0x2a13e60, 0xc00104b140})
	github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.1/helper/schema/field_writer_map.go:222 +0x312
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*MapFieldWriter).set(0xc001029758, {0xc0004eab40, 0x6, 0x6}, {0x2a13e60, 0xc00104b140})
	github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.1/helper/schema/field_writer_map.go:109 +0x11c
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*MapFieldWriter).setList.func1(...)
	github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.1/helper/schema/field_writer_map.go:122
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*MapFieldWriter).setList(0xc001029758, {0xc000138870?, 0x5, 0x5}, {0x288a0a0, 0xc001029878})
	github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.1/helper/schema/field_writer_map.go:142 +0x52a
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*MapFieldWriter).set(0xc001029758, {0xc000138870, 0x5, 0x5}, {0x288a0a0, 0xc001029878})
	github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.1/helper/schema/field_writer_map.go:103 +0xa5
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*MapFieldWriter).setObject(0xc00038ff00?, {0xc00038ff00?, 0x4, 0x4}, {0x2a13e60, 0xc00104b0b0})
	github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.1/helper/schema/field_writer_map.go:222 +0x312
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*MapFieldWriter).set(0xc001029758, {0xc00038ff00, 0x4, 0x4}, {0x2a13e60, 0xc00104b0b0})
	github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.1/helper/schema/field_writer_map.go:109 +0x11c
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*MapFieldWriter).setList.func1(...)
	github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.1/helper/schema/field_writer_map.go:122
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*MapFieldWriter).setList(0xc001029758, {0xc00104b290?, 0x3, 0x3}, {0x288a0a0, 0xc001029890})
	github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.1/helper/schema/field_writer_map.go:142 +0x52a
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*MapFieldWriter).set(0xc001029758, {0xc00104b290, 0x3, 0x3}, {0x288a0a0, 0xc001029890})
	github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.1/helper/schema/field_writer_map.go:103 +0xa5
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*MapFieldWriter).setObject(0xc0005363a0?, {0xc0005363a0?, 0x2, 0x2}, {0x2a13e60, 0xc00104b050})
	github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.1/helper/schema/field_writer_map.go:222 +0x312
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*MapFieldWriter).set(0xc001029758, {0xc0005363a0, 0x2, 0x2}, {0x2a13e60, 0xc00104b050})
	github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.1/helper/schema/field_writer_map.go:109 +0x11c
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*MapFieldWriter).setList.func1(...)
	github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.1/helper/schema/field_writer_map.go:122
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*MapFieldWriter).setList(0xc001029758, {0xc0003d7e00?, 0x1, 0x1}, {0x288a0a0, 0xc0010298f0})
	github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.1/helper/schema/field_writer_map.go:142 +0x52a
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*MapFieldWriter).set(0xc001029758, {0xc0003d7e00, 0x1, 0x1}, {0x288a0a0, 0xc0010298f0})
	github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.1/helper/schema/field_writer_map.go:103 +0xa5
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*MapFieldWriter).WriteField(0xc001029758, {0xc0003d7e00, 0x1, 0x1}, {0x288a0a0, 0xc0010298f0})
	github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.1/helper/schema/field_writer_map.go:89 +0x3f9
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*ResourceData).Set(0xc001146400, {0x2eca8ef, 0x4}, {0x288a0a0, 0xc0010298f0})
	github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.1/helper/schema/resource_data.go:227 +0x210
github.com/oboukili/terraform-provider-argocd/argocd.flattenApplication(0xc000581c00, 0x34d1aa0?)
	github.com/oboukili/terraform-provider-argocd/argocd/structure_application.go:506 +0x178
github.com/oboukili/terraform-provider-argocd/argocd.resourceArgoCDApplicationRead({0x34d1aa0, 0xc00060cae0}, 0xc001146400, {0x2c24d00?, 0xc000474460})
	github.com/oboukili/terraform-provider-argocd/argocd/resource_argocd_application.go:356 +0x2a5
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).read(0xc001058380, {0x34d1ad8, 0xc0011db380}, 0xd?, {0x2c24d00, 0xc000474460})
	github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.1/helper/schema/resource.go:724 +0x12e
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).RefreshWithoutUpgrade(0xc001058380, {0x34d1ad8, 0xc0011db380}, 0xc00113e0d0, {0x2c24d00, 0xc000474460})
	github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.1/helper/schema/resource.go:1015 +0x585
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).ReadResource(0xc001029338, {0x34d1ad8?, 0xc0011db260?}, 0xc0005b5040)
	github.com/hashicorp/terraform-plugin-sdk/v2@v2.24.1/helper/schema/grpc_provider.go:613 +0x4a5
github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.(*server).ReadResource(0xc0005288c0, {0x34d1ad8?, 0xc0011daa80?}, 0xc0002c0540)
	github.com/hashicorp/terraform-plugin-go@v0.14.1/tfprotov5/tf5server/server.go:748 +0x4b1
github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_ReadResource_Handler({0x2d5a580?, 0xc0005288c0}, {0x34d1ad8, 0xc0011daa80}, 0xc0005d4460, 0x0)
	github.com/hashicorp/terraform-plugin-go@v0.14.1/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:349 +0x170
google.golang.org/grpc.(*Server).processUnaryRPC(0xc000036780, {0x34de860, 0xc00056d6c0}, 0xc0011d9560, 0xc001071c50, 0x4d201b0, 0x0)
	google.golang.org/grpc@v1.51.0/server.go:1340 +0xd23
google.golang.org/grpc.(*Server).handleStream(0xc000036780, {0x34de860, 0xc00056d6c0}, 0xc0011d9560, 0x0)
	google.golang.org/grpc@v1.51.0/server.go:1713 +0xa2f
google.golang.org/grpc.(*Server).serveStreams.func1.2()
	google.golang.org/grpc@v1.51.0/server.go:965 +0x98
created by google.golang.org/grpc.(*Server).serveStreams.func1
	google.golang.org/grpc@v1.51.0/server.go:963 +0x28a

Error: The terraform-provider-argocd_v5.3.0 plugin crashed!
@josecsotomorales josecsotomorales added the bug Something isn't working label May 15, 2023
@onematchfox
Copy link
Collaborator

onematchfox commented May 15, 2023

Hi @josecsotomorales,

Unfortunately your description is not sufficient to enable us to figure out what is causing this issue. Please could you provide the Terraform configuration/code for the affected argocd_application resource?

@onematchfox onematchfox added the needs-more-info Waiting on additional information from issue/PR reporter. label May 15, 2023
@onematchfox onematchfox changed the title Plugin crash with set item just set doesn't exist Plugin crash with "set item just set doesn't exist" May 15, 2023
@josecsotomorales
Copy link
Author

Hey @onematchfox! Thanks for your quick response! Posting the TF file I'm using, hope it helps:

data aws_eks_cluster "target_cluster" {
  name = var.eks_cluster_name
}

resource "argocd_application" "helm" {
  metadata {
    name      = var.env
    namespace = "argocd"
    labels = {
      environment = var.env
    }
  }

  wait = true


  spec {
    source {
      repo_url        = var.helm_qualytics_repo_url
      chart           = var.helm_chart_name
      target_revision = var.target_helm_chart_revision
      helm {
        parameter {
          name  = "spec.destination.server"
          value = data.aws_eks_cluster.target_cluster.endpoint
        }

        parameter {
          name  = "spec.gitRepo.trackingBranch"
          value = var.client_config_manifests_branch
        }

        parameter {
          name = "spec.clientChartName"
          value = var.client_chart_name
        }
        release_name = var.env
      }
    }

    sync_policy {
      automated {
        prune = false
        self_heal = false
      }
    }

    destination {
      server    = "https://kubernetes.default.svc"
      namespace = "argocd"
    }
  }
}

I'm using this module dynamically for multiple deployments, and it works for some deployments ... weird.

@onematchfox onematchfox removed the needs-more-info Waiting on additional information from issue/PR reporter. label May 15, 2023
@onematchfox
Copy link
Collaborator

Hi @josecsotomorales,

Thanks for supplying the config. Unfortunately, even with this, I have been unable to reproduce this error. There also doesn't seem to be much information out there regarding why this may occur. I did find a similar issue in the Kubernetes provider, which resulted from a block containing a single attribute where the value was set to null. However, even if I play around with your setting your variables to null, I still can't reproduce your specific error.

I'm using this module dynamically for multiple deployments, and it works for some deployments ... weird.

If it works for some and not others, then this does feel like a configuration issue. Are you able to reproduce this consistently?

@onematchfox onematchfox added the cannot-reproduce Unable to reproduce issue. label May 16, 2023
@josecsotomorales
Copy link
Author

Thanks for helping on this @onematchfox! Of the 10+ deployments I have 2 are getting this failure consistently, will keep investigating and post an update here on my findings, will close this issue for now 🚀

@onematchfox
Copy link
Collaborator

Thanks for helping on this @onematchfox! Of the 10+ deployments I have 2 are getting this failure consistently, will keep investigating and post an update here on my findings, will close this issue for now 🚀

Ok great. I look forward to hearing more. If you do manage to figure out what is causing this then I'd love to make sure we fix the issue.

@mkilchhofer
Copy link
Collaborator

mkilchhofer commented Nov 7, 2023

I now also see this error in our env at @swisspost.

╷
│ Error: Plugin did not respond
│ 
│ The plugin encountered an error, and failed to respond to the
│ plugin6.(*GRPCProvider).ValidateResourceConfig call. The plugin logs may
│ contain more details.
╵
╷
│ Error: Plugin did not respond
│ 
│ The plugin encountered an error, and failed to respond to the
│ plugin6.(*GRPCProvider).ValidateResourceConfig call. The plugin logs may
│ contain more details.
╵
╷
│ Error: Plugin did not respond
│ 
│ The plugin encountered an error, and failed to respond to the
│ plugin6.(*GRPCProvider).ReadResource call. The plugin logs may contain more
│ details.
╵
╷
│ Error: Plugin did not respond
│ 
│ The plugin encountered an error, and failed to respond to the
│ plugin6.(*GRPCProvider).ReadResource call. The plugin logs may contain more
│ details.
╵
╷
│ Error: Plugin did not respond
│ 
│ The plugin encountered an error, and failed to respond to the
│ plugin6.(*GRPCProvider).ReadResource call. The plugin logs may contain more
│ details.
╵
╷
│ Error: Plugin did not respond
│ 
│ The plugin encountered an error, and failed to respond to the
│ plugin6.(*GRPCProvider).ReadResource call. The plugin logs may contain more
│ details.
╵
╷
│ Error: Plugin did not respond
│ 
│ The plugin encountered an error, and failed to respond to the
│ plugin6.(*GRPCProvider).ValidateResourceConfig call. The plugin logs may
│ contain more details.
╵
╷
│ Error: Request cancelled
│ 
│ The plugin6.(*GRPCProvider).UpgradeResourceState request was cancelled.
╵

Stack trace from the terraform-provider-argocd_v6.0.3 plugin:

panic: set item just set doesn't exist

goroutine 470 [running]:
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*MapFieldWriter).setSet(0xc000f93950, {0xc000e23260, 0x3, 0x3}, {0x29ffda0, 0xc000f93b78}, 0xc0008d63c0)
	github.com/hashicorp/terraform-plugin-sdk/v2@v2.26.1/helper/schema/field_writer_map.go:327 +0x959
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*MapFieldWriter).set(0xc000f93950, {0xc000e23260, 0x3, 0x3}, {0x29ffda0, 0xc000f93b78})
	github.com/hashicorp/terraform-plugin-sdk/v2@v2.26.1/helper/schema/field_writer_map.go:107 +0x14c
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*MapFieldWriter).setObject(0xc0003e14e0?, {0xc0003e14e0?, 0x2, 0x2}, {0x2b943a0, 0xc000e22ff0})
	github.com/hashicorp/terraform-plugin-sdk/v2@v2.26.1/helper/schema/field_writer_map.go:222 +0x312
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*MapFieldWriter).set(0xc000f93950, {0xc0003e14e0, 0x2, 0x2}, {0x2b943a0, 0xc000e22ff0})
	github.com/hashicorp/terraform-plugin-sdk/v2@v2.26.1/helper/schema/field_writer_map.go:109 +0x11c
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*MapFieldWriter).setList.func1(...)
	github.com/hashicorp/terraform-plugin-sdk/v2@v2.26.1/helper/schema/field_writer_map.go:122
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*MapFieldWriter).setList(0xc000f93950, {0xc002148bf0?, 0x1, 0x1}, {0x29ffd20, 0xc000f93c50})
	github.com/hashicorp/terraform-plugin-sdk/v2@v2.26.1/helper/schema/field_writer_map.go:142 +0x52a
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*MapFieldWriter).set(0xc000f93950, {0xc002148bf0, 0x1, 0x1}, {0x29ffd20, 0xc000f93c50})
	github.com/hashicorp/terraform-plugin-sdk/v2@v2.26.1/helper/schema/field_writer_map.go:103 +0xa5
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*MapFieldWriter).WriteField(0xc000f93950, {0xc002148bf0, 0x1, 0x1}, {0x29ffd20, 0xc000f93c50})
	github.com/hashicorp/terraform-plugin-sdk/v2@v2.26.1/helper/schema/field_writer_map.go:89 +0x3f9
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*ResourceData).Set(0xc002507800, {0x306e4f8, 0x4}, {0x29ffd20, 0xc000f93c50})
	github.com/hashicorp/terraform-plugin-sdk/v2@v2.26.1/helper/schema/resource_data.go:227 +0x210
github.com/oboukili/terraform-provider-argocd/argocd.flattenApplication(0xc00127d400, 0x36b9500?)
	github.com/oboukili/terraform-provider-argocd/argocd/structure_application.go:536 +0x288
github.com/oboukili/terraform-provider-argocd/argocd.resourceArgoCDApplicationRead({0x36b9500, 0xc002539e00}, 0xc002507800, {0x2e595e0?, 0xc0003a4c80})
	github.com/oboukili/terraform-provider-argocd/argocd/resource_argocd_application.go:229 +0x2a5
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).read(0xc000900c40, {0x36b9538, 0xc001908b70}, 0xd?, {0x2e595e0, 0xc0003a4c80})
	github.com/hashicorp/terraform-plugin-sdk/v2@v2.26.1/helper/schema/resource.go:724 +0x12e
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).RefreshWithoutUpgrade(0xc000900c40, {0x36b9538, 0xc001908b70}, 0xc001f89790, {0x2e595e0, 0xc0003a4c80})
	github.com/hashicorp/terraform-plugin-sdk/v2@v2.26.1/helper/schema/resource.go:1015 +0x585
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).ReadResource(0xc000402018, {0x36b9538?, 0xc0019089f0?}, 0xc00142bd40)
	github.com/hashicorp/terraform-plugin-sdk/v2@v2.26.1/helper/schema/grpc_provider.go:613 +0x4a5
github.com/hashicorp/terraform-plugin-mux/tf5to6server.v5tov6Server.ReadResource({{0x36cd190?, 0xc000402018?}}, {0x36b9538?, 0xc0019089f0?}, 0xc00142bbc0?)
	github.com/hashicorp/terraform-plugin-mux@v0.10.0/tf5to6server/tf5to6server.go:105 +0x242
github.com/hashicorp/terraform-plugin-mux/tf6muxserver.muxServer.ReadResource({0xc0008f1680, 0xc0008f16e0, {0xc000fa0b20, 0x2, 0x2}, {0x0, 0x0, 0x0}, {0x0, 0x0, ...}, ...}, ...)
	github.com/hashicorp/terraform-plugin-mux@v0.10.0/tf6muxserver/mux_server_ReadResource.go:26 +0x102
github.com/hashicorp/terraform-plugin-go/tfprotov6/tf6server.(*server).ReadResource(0xc0008577c0, {0x36b9538?, 0xc00128fe00?}, 0xc000548120)
	github.com/hashicorp/terraform-plugin-go@v0.15.0/tfprotov6/tf6server/server.go:746 +0x4b1
github.com/hashicorp/terraform-plugin-go/tfprotov6/internal/tfplugin6._Provider_ReadResource_Handler({0x2f101a0?, 0xc0008577c0}, {0x36b9538, 0xc00128fe00}, 0xc0009fb730, 0x0)
	github.com/hashicorp/terraform-plugin-go@v0.15.0/tfprotov6/internal/tfplugin6/tfplugin6_grpc.pb.go:383 +0x170
google.golang.org/grpc.(*Server).processUnaryRPC(0xc0006443c0, {0x36c7680, 0xc000c4dba0}, 0xc0014c0c60, 0xc001070de0, 0x504e170, 0x0)
	google.golang.org/grpc@v1.54.0/server.go:1345 +0xdf0
google.golang.org/grpc.(*Server).handleStream(0xc0006443c0, {0x36c7680, 0xc000c4dba0}, 0xc0014c0c60, 0x0)
	google.golang.org/grpc@v1.54.0/server.go:1722 +0xa2f
google.golang.org/grpc.(*Server).serveStreams.func1.2()
	google.golang.org/grpc@v1.54.0/server.go:966 +0x98
created by google.golang.org/grpc.(*Server).serveStreams.func1
	google.golang.org/grpc@v1.54.0/server.go:964 +0x28a

Error: The terraform-provider-argocd_v6.0.3 plugin crashed!

This is always indicative of a bug within the plugin. It would be immensely
helpful if you could report the crash with the plugin's maintainers so that it
can be fixed. The output above should help diagnose the issue.

Since I only see this error on one of our ~45 k8s clusters I suspect something in our env (it's the dev/sandbox cluster where every engineer is doing nasty stuff). I will keep you posted if I find out something.

@mkilchhofer
Copy link
Collaborator

ahaaaa 💡 i reproduced it.
Someone in my team (probably me) added a INFO field to an application:
image

Then the INFO field was removed inside the UI but it was still present in the App spec:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  creationTimestamp: "2023-03-09T10:18:25Z"
  generation: 147673
  name: aqua
  namespace: argocd
  resourceVersion: "xyz"
  uid: xyz
spec:
  destination:
    namespace: xyz
    server: https://kubernetes.default.svc
  ignoreDifferences:
  - group: apiextensions.k8s.io
    # ...
  info:
  - name: ""  # <---------------------------------------------------- see here
    value: "" # <---------------------------------------------------- see here
  project: swisspost-xyz
  revisionHistoryLimit: 10
  source:
    helm:
      valueFiles:
      - values.yaml
      # ...

As soon as I edit the manifest via kubectl and remove these three lines

  info:
  - name: ""  # <---------------------------------------------------- see here
    value: "" # <---------------------------------------------------- see here

, everything is working again. Maybe if you have time @onematchfox you could have a look as you are familiar with the code base :-)

@onematchfox
Copy link
Collaborator

ahaaaa 💡 i reproduced it.

Nicely done sir! And thanks for the PR. Will try to take a look tomorrow.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working cannot-reproduce Unable to reproduce issue.
Projects
None yet
3 participants