ignition: extend cache to keep files on disk to keep data intact between invocations #50
Conversation
- go get github.com/peterbourgon/diskv/v3 - go mod tidy && go mod vendor
ghost
added
…een invocations With errors like issue 116 [1], terraform restarts the provider server. And with terraform turning on `AutoMTLS` for all plugins [2], the client cannot reattach to already running provider server [3]. Provider's memory only caches creates errors like `invalid file \"49e2cc7ba023d87070d9b49a0c8f02ff0f63b7f61b390fbce8f64b011bd42ed7\", unknown file id` because the ignition_config resource tried to retrived a file id stored in cache before provider server was restarted. This stores the objects in `diskv` store in users' cache directory, retireving it when the memory cache is not populated. The in-memory cache keeps providing the no overhead access to objects. [1]: hashicorp/go-plugin#116 [2]: hashicorp/terraform#19560 [3]: https://github.com/hashicorp/go-plugin/blob/5692942914bbdbc03558fde936b1f0bc2af365be/client.go#L203-L204
…ted 0.12 terraform source This commit changes the source for 2 providers `openstack` and `ignition` because: - openstack PR 753 [1] moves the openstack provider to use 0.12 terraform libraries. - ignition PR 50 [2] is required because old implementation of ignition provider stores objects in memory and with terraform 0.12 errors regarding to reading from plugins servers forces new procress to be created due to AutoMTLS. Check the PR description for more info [3]. [1]: https://github.com/terraform-providers/terraform-provider-openstack/pull/753 [2]: hashicorp/terraform-provider-ignition#50
…ted 0.12 terraform source This commit changes the source for 2 providers `openstack` and `ignition` because: - openstack PR 753 [1] moves the openstack provider to use 0.12 terraform libraries. - ignition PR 50 [2] is required because old implementation of ignition provider stores objects in memory and with terraform 0.12 errors regarding to reading from plugins servers forces new procress to be created due to AutoMTLS. Check the PR description for more info [3]. [1]: https://github.com/terraform-providers/terraform-provider-openstack/pull/753 [2]: hashicorp/terraform-provider-ignition#50
| } | ||
| } | ||
|
|
||
| func providerConfigure(d *schema.ResourceData) (interface{}, error) { | ||
| basePath := filepath.Join(os.TempDir(), "terraform-provider-ignition") |
There was a problem hiding this comment.
Were it possible for more than one Terraform run to be underway at the same time, they would collide here, trying to write to the same files under this path. Consider using ioutil.TempDir instead.
| func providerConfigure(d *schema.ResourceData) (interface{}, error) { | ||
| basePath := filepath.Join(os.TempDir(), "terraform-provider-ignition") | ||
|
|
||
| globalCache.d = diskv.New(diskv.Options{ |
There was a problem hiding this comment.
Given that hashicorp/terraform#6258 remains open, is it correct that we never delete this directory and the files within it? (There is a (*Diskv).EraseAll method.)
|
I tried this patch, and found that it worked with Terraform version 0.12.7. Per my comment above in file ignition/provider.go, there's no cleanup of the files written by diskv, so it appears that they'll accumulate without bound over time, until rebooting the machine purges the temporary directory. Can you think of a tenable approach to curbing that growth? I take it we can't purge these files when the program starts, because we may be restarting in the interest of recovering the cache left behind from a previous—and very recent—run of the program. |
|
Now that #56 is committed, it doesn't seem likely that we'll accept this patch, given that it's a different approach to solve the same problem. Shall we abandon this patch? |
|
I believe this issue can be closed now. |
With errors like issue 116 1, terraform restarts the provider server. And with terraform turning
on
AutoMTLSfor all plugins 2, the client cannot reattach to already running provider server 3.Provider's memory only caches creates errors like
invalid file \"49e2cc7ba023d87070d9b49a0c8f02ff0f63b7f61b390fbce8f64b011bd42ed7\", unknown file idbecause the ignition_config resource tried to retrived a file id stored in cache before provider server was restarted.
This stores the objects in
diskvstore in users' cache directory, retireving it when the memory cache is not populated.The in-memory cache keeps providing the no overhead access to objects.