feat: add disk stats for datastore #1896

nikfot · 2023-05-05T11:51:36Z

Description

There is currently no straightforward way to choose a datastore based on usage when creating a VM. This way more criteria can be added to choosing a datastore for a VM. For example, select a datastore if the total capacity is greater than a value or free space is greater than a value. I intend to also add a sorting of all datastores as part of this PR.

Add total capacity and free space for datastore. This way more criteria can be added to datastore selection

Acceptance tests

Have you added an acceptance test for the functionality being added?
No, will be added after first feedback.
Have you run the acceptance tests on this branch?
Yes

Output from acceptance testing:

$ make testacc TESTARGS="-run=TestAccDataSourceVSphereDatastoreStats_basic"                    
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test $(go list ./... |grep -v 'vendor') -v -run=TestAccDataSourceVSphereDatastoreStats_basic -timeout 240m
?       github.com/hashicorp/terraform-provider-vsphere [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/administrationroles    [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/clustercomputeresource  [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/computeresource [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/contentlibrary  [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/customattribute [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/datacenter      [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/datastore       [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/dvportgroup     [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/envbrowse       [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/folder  [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/hostsystem      [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/network [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/nsx     [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/ovfdeploy       [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/provider        [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/resourcepool    [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/spbm    [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/storagepod      [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/structure       [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/testhelper      [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/utils   [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/vappcontainer   [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/virtualmachine  [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/vsanclient      [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/vsansystem      [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/vmworkflow     [no test files]
=== RUN   TestAccDataSourceVSphereDatastoreStats_basic
true true
true true
true true
true true
--- PASS: TestAccDataSourceVSphereDatastoreStats_basic (29.63s)
PASS
ok      github.com/hashicorp/terraform-provider-vsphere/vsphere 29.649s
testing: warning: no tests to run
PASS
ok      github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/viapi   (cached) [no tests to run]
testing: warning: no tests to run
PASS
ok      github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/virtualdisk     (cached) [no tests to run]
testing: warning: no tests to run
PASS
ok      github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/virtualdevice  (cached) [no tests to run]

 make testacc TESTARGS="-run=TestAccDataSourceVSphereDatastore_getStats"           
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test $(go list ./... |grep -v 'vendor') -v -run=TestAccDataSourceVSphereDatastore_getStats -timeout 240m
?       github.com/hashicorp/terraform-provider-vsphere [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/administrationroles    [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/clustercomputeresource  [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/computeresource [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/contentlibrary  [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/customattribute [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/datacenter      [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/datastore       [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/dvportgroup     [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/envbrowse       [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/folder  [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/hostsystem      [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/network [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/nsx     [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/ovfdeploy       [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/provider        [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/resourcepool    [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/spbm    [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/storagepod      [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/structure       [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/testhelper      [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/utils   [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/vappcontainer   [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/virtualmachine  [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/vsanclient      [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/vsansystem      [no test files]
?       github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/vmworkflow     [no test files]
=== RUN   TestAccDataSourceVSphereDatastore_getStats
--- PASS: TestAccDataSourceVSphereDatastore_getStats (16.39s)
PASS
ok      github.com/hashicorp/terraform-provider-vsphere/vsphere 16.412s
testing: warning: no tests to run
PASS
ok      github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/viapi   (cached) [no tests to run]
testing: warning: no tests to run
PASS
ok      github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/helper/virtualdisk     (cached) [no tests to run]
testing: warning: no tests to run
PASS
ok      github.com/hashicorp/terraform-provider-vsphere/vsphere/internal/virtualdevice  (cached) [no tests to run]

Release Note

Release note for CHANGELOG:

- `data/vsphere_datastore_stats` Add vsphere datastore stats to report total capacity and free space of datastores
- `data/vsphere_datastore`  Add stats for a single datastore

References

Close #1893

hashicorp-cla · 2023-05-05T11:51:40Z

All committers have signed the CLA.

tenthirtyam · 2023-05-05T16:34:57Z

Marked pull request as a draft until Ready for Review.

nikfot · 2023-05-11T10:33:37Z

@tenthirtyam I completed the feature, however I had trouble running the acceptance tests with the following error:

=== RUN   TestAccDataSourceVSphereDatastoreStats_basic
    data_source_vsphere_datastore_stats_test.go:17: Step 1/1 error: Error running pre-apply refresh: exit status 1
        
        Error: Missing required argument
        
          on terraform_plugin_test.tf line 1, in provider "vsphere":
           1: provider "vsphere" {}
        
        The argument "user" is required, but no definition was found.
        
        Error: Missing required argument
        
          on terraform_plugin_test.tf line 1, in provider "vsphere":
           1: provider "vsphere" {}
        
        The argument "password" is required, but no definition was found.
--- FAIL: TestAccDataSourceVSphereDatastoreStats_basic (27.37s)

Although I set the relevant env vars. I don't know if there is something I am missing about the unit tests setup.
Please let me know.
Thanks.

appilon · 2023-05-11T20:33:07Z

@nikfot Via environment variables you need to configure the username/password and Address of the vsphere server:

export VSPHERE_USER="user@server"
export VSPHERE_PASSWORD="password"
export VSPHERE_SERVER="192.168.0.2"

Please keep in mind this will create and destroy resources against the vCenter instance, so make sure its done against a non-production environment

nikfot · 2023-05-12T10:48:32Z

@appilon thanks! i had the TF_VAR_ set to user etc and I thought it would be enough.

Anyway, I ran the acceptance tests successfully.

Please proceed with the review.

nikfot · 2023-05-17T07:59:03Z

@tenthirtyam any update on this one?

nikfot · 2023-06-14T13:47:12Z

Hello,
Are there any updates on this PR?
Thanks!

tenthirtyam · 2023-06-14T13:58:59Z

Are there any updates on this PR?

Unfortunately, not at this time due to other priorities.

nikfot · 2023-06-19T09:22:45Z

Are there any updates on this PR?

Unfortunately, not at this time due to other priorities.

@tenthirtyam that's totally understandable, but is there a standard process about it? How and when will it be discussed etc. This addition would be really helpfull since me and a bunch of other people will use it in vmware production environments.

I also have more things that I would like to contribute, but if they are not going to be concidered it would be in vain. I would also like to contribute more to the community as far as issues and more are concidered and the current PR is on a "to know us better basis", that is why I am asking.

nikfot · 2023-12-15T11:29:46Z

Hello,

Since I have no remaining task on my side, please give me some feedback on this pull request.
@vasilsatanasov @appilon

Thanks,
Nikos

vasilsatanasov · 2023-12-15T12:30:14Z

Hi @nikfot IMHO it is good to go into the next release. We won't have any more releases this year due to stability reasons.

vasilsatanasov

LGTM!

tenthirtyam

Actually, just a quick look and I notice that the data source documentation needs to be updates along with this change.

Add total capacity and free space for datastore. This way more criteria can be added to datastore selection Add list of datastores with free space and total capacity. Enhance documentation for datastore data source. Add documentation for datastore stats data source. Signed-off-by: nikfot <nik_fot@hotmail.gr>

nikfot · 2024-01-21T17:11:57Z

Hello @tenthirtyam && @vasilsatanasov and belated wishes for a happy 2024!

@tenthirtyam I added the necessary documentation. Please check it out and let me know if it is good to go for the next release.

Thanks,
Nikos

nikfot · 2024-01-25T08:12:42Z

Hello @tenthirtyam! Did you find time to review tge additions?

nikfot · 2024-01-30T13:01:16Z

@tenthirtyam @vasilsatanasov could someone provide a feedback on this one please?

AntonisKatsifas · 2024-02-05T10:31:46Z

any news on this one? will it be on the next release?

tenthirtyam

LGTM

nikfot · 2024-02-12T17:16:37Z

LGTM

thanks @tenthirtyam . will these changes be on the 2.7.0 release?

tenthirtyam · 2024-02-12T17:28:48Z

Yes.

github-actions · 2024-03-06T16:46:31Z

This functionality has been released in v2.7.0 of the Terraform Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

github-actions · 2024-04-06T02:03:59Z

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

nikfot requested a review from a team as a code owner May 5, 2023 11:51

github-actions bot added provider Type: Provider size/xs Relative Sizing: Extra-Small labels May 5, 2023

nikfot force-pushed the add-datastore-capacity branch 4 times, most recently from 1c5aa7c to 82c6d6e Compare May 5, 2023 14:07

github-actions bot added the size/m Relative Sizing: Medium label May 5, 2023

tenthirtyam marked this pull request as draft May 5, 2023 16:33

tenthirtyam assigned nikfot May 5, 2023

tenthirtyam added enhancement Type: Enhancement and removed size/xs Relative Sizing: Extra-Small labels May 5, 2023

tenthirtyam added this to the Backlog milestone May 5, 2023

tenthirtyam changed the title ~~[WIP]: Add disk stats for datastore~~ feat: Add disk stats for datastore May 8, 2023

tenthirtyam changed the title ~~feat: Add disk stats for datastore~~ feat: add disk stats for datastore May 8, 2023

nikfot force-pushed the add-datastore-capacity branch from 82c6d6e to 97e4bfc Compare May 11, 2023 10:31

github-actions bot added the size/l Relative Sizing: Large label May 11, 2023

nikfot force-pushed the add-datastore-capacity branch from 97e4bfc to 0d979ae Compare May 11, 2023 10:34

nikfot force-pushed the add-datastore-capacity branch from 0d979ae to f1f6787 Compare May 12, 2023 10:25

nikfot marked this pull request as ready for review May 12, 2023 10:48

nikfot force-pushed the add-datastore-capacity branch from f1f6787 to 6b7653a Compare May 12, 2023 10:50

tenthirtyam added the merge-conflict Merge Conflict label Oct 19, 2023

tenthirtyam marked this pull request as draft October 19, 2023 16:36

nikfot force-pushed the add-datastore-capacity branch from 6b7653a to 969db34 Compare October 25, 2023 12:36

tenthirtyam modified the milestones: v2.6.0, Backlog Nov 10, 2023

nikfot marked this pull request as ready for review December 11, 2023 12:24

vasilsatanasov approved these changes Dec 15, 2023

View reviewed changes

This comment was marked as outdated.

Sign in to view

tenthirtyam requested changes Dec 15, 2023

View reviewed changes

nikfot force-pushed the add-datastore-capacity branch from 969db34 to 61276fc Compare January 21, 2024 17:09

tenthirtyam modified the milestones: Backlog, v2.7.0 Feb 12, 2024

tenthirtyam removed the merge-conflict Merge Conflict label Feb 12, 2024

tenthirtyam approved these changes Feb 12, 2024

View reviewed changes

tenthirtyam requested a review from iBrandyJackson February 12, 2024 17:29

iBrandyJackson approved these changes Feb 12, 2024

View reviewed changes

tenthirtyam merged commit 8370be8 into hashicorp:main Feb 12, 2024
4 checks passed

nikfot deleted the add-datastore-capacity branch February 13, 2024 09:27

github-actions bot locked as resolved and limited conversation to collaborators Apr 6, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add disk stats for datastore #1896

feat: add disk stats for datastore #1896

nikfot commented May 5, 2023 •

edited

Loading

hashicorp-cla commented May 5, 2023 •

edited

Loading

tenthirtyam commented May 5, 2023

nikfot commented May 11, 2023

appilon commented May 11, 2023 •

edited

Loading

nikfot commented May 12, 2023

nikfot commented May 17, 2023

nikfot commented Jun 14, 2023

tenthirtyam commented Jun 14, 2023

nikfot commented Jun 19, 2023

nikfot commented Dec 15, 2023

vasilsatanasov commented Dec 15, 2023

vasilsatanasov left a comment

This comment was marked as outdated.

tenthirtyam left a comment

nikfot commented Jan 21, 2024

nikfot commented Jan 25, 2024

nikfot commented Jan 30, 2024

AntonisKatsifas commented Feb 5, 2024

tenthirtyam left a comment

nikfot commented Feb 12, 2024

tenthirtyam commented Feb 12, 2024

github-actions bot commented Mar 6, 2024

github-actions bot commented Apr 6, 2024

feat: add disk stats for datastore #1896

feat: add disk stats for datastore #1896

Conversation

nikfot commented May 5, 2023 • edited Loading

Description

Acceptance tests

Release Note

References

hashicorp-cla commented May 5, 2023 • edited Loading

tenthirtyam commented May 5, 2023

nikfot commented May 11, 2023

appilon commented May 11, 2023 • edited Loading

nikfot commented May 12, 2023

nikfot commented May 17, 2023

nikfot commented Jun 14, 2023

tenthirtyam commented Jun 14, 2023

nikfot commented Jun 19, 2023

nikfot commented Dec 15, 2023

vasilsatanasov commented Dec 15, 2023

vasilsatanasov left a comment

Choose a reason for hiding this comment

This comment was marked as outdated.

tenthirtyam left a comment

Choose a reason for hiding this comment

nikfot commented Jan 21, 2024

nikfot commented Jan 25, 2024

nikfot commented Jan 30, 2024

AntonisKatsifas commented Feb 5, 2024

tenthirtyam left a comment

Choose a reason for hiding this comment

nikfot commented Feb 12, 2024

tenthirtyam commented Feb 12, 2024

github-actions bot commented Mar 6, 2024

github-actions bot commented Apr 6, 2024

nikfot commented May 5, 2023 •

edited

Loading

hashicorp-cla commented May 5, 2023 •

edited

Loading

appilon commented May 11, 2023 •

edited

Loading