FR: Improve Test Coverage: A Test Plan

[bigtedde]

The problem

Vorta's test coverage is currently 72% according to the coverage tool. I would like to increase that coverage to 80% or higher by parameterizing some of the current tests, as well as creating additional unit tests.

Requested Solution

pytest --cov=vorta --cov-report term-missing

This command yields a coverage report detailing the current code coverage of 72% as well as the lines in each file that are being missed by the current set of tests. Using this coverage report, I created a simple spreadsheet detailing all files with a code coverage of <70% and the areas of opportunity for additional tests. These files offer the greatest opportunity to increase code coverage, specifically through unit testing of currently untested functions/methods.

Additional context

@jetchirag is currently working with the test suite on issues #1716 and #1711, so I will wait until after he is finished before progressing to far with this project.

[m3nu]

Our testing expert has entered the room. He honed his expertise on thousands of Borg tests. So this is easy for him. 😄

[diivi]

@bigtedde I might need your help with the tests for #1742 too 😉, and thanks for the great work!

[jetchirag]

@jetchirag is currently working with the test suite on issues #1716 and #1711, so I will wait until after he is finished before progressing to far with this project.

I am mostly done with the task. Currently all tests are passing too but they are mostly identical to the unit tests so coverage won't really be effected much except some functions like local folder selector, extract etc. It would be great if you can have a look them though and we can cover the part which can't be included in the unit tests there.

[bigtedde]

I might need your help with the tests for #1742 too 😉, and thanks for the great work!

@diivi I would be happy to help with testing your exclude GUI. I took a look at your PR - maybe there is a need for path/pattern testing?

[diivi]

maybe there is a need for path/pattern testing?

I'll probably just test all 3 tabs and if they alter the preview tab correctly.

And what gets passed to Borg.

After that everything should be handled by Borg, so I don't think testing if a particular pattern works or not should be necessary.

[bigtedde]

I am mostly done with the task. Currently all tests are passing too but they are mostly identical to the unit tests so coverage won't really be effected much except some functions like local folder selector, extract etc. It would be great if you can have a look them though and we can cover the part which can't be included in the unit tests there.

@jetchirag Great to hear that all tests are passing, and it sounds like your close to completing! I looked through your code from PR #1716 and if I am understanding it correctly you have been editing the tests to work on a live Borg binary instead of the hardcoded json outputs, and separating each file file into integration tests vs unit tests. I am really thankful for you putting in this work, as I think you are making the test suite much more approachable, orderly, and modular.

I would like to wait until your PR gets merged since it is essentially a new foundation for the test suite, and I would like to build upon it by filling in code coverage with additional unit tests and parameterizations of current ones. If you have any ideas of coverage that is most urgently needed from your time working with the test suite , I'd appreciate your insight :)

[bigtedde]

I'll probably just test all 3 tabs and if they alter the preview tab correctly.

And what gets passed to Borg.

I would be happy to help with this :)

[bigtedde]

After the most recent test suite merges into master our total code coverage now sits at 78%.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[github-actions]

This issue was closed because it has been stalled for 7 days with no activity.

[m3nu]

(Was already completed actually)