ci: add pytest support on windows; change: add meltano elt guidance for windows users

[visch]

Closes #5940

Old MR from Gitlab https://gitlab.com/meltano/meltano/-/merge_requests/2639/diffs

(Note this has been rebased since this MR)

Pipeline is running here https://gitlab.com/vischous/meltano/-/merge_requests/3/pipelines

[netlify]

👷 Deploy request for meltano pending review.

Visit the deploys page to approve it

Name	Link
🔨 Latest commit	b029791

[visch]

How do you feel about punting that to #3444 I added a comment there? I think that issue is going to end up being a bunch of sub issues addressing individual failures, and one of those should be switching to xfail for everything.

Works for me, though an alternative that lets us use xfail and avoid the having the process hang would be adding something like this at the start of each of these failing tests:

def test_something(...):
    if platform.system() == "Windows":
        pytest.xfail("reason...")
    # The original body of the test function continues here

How do you feel about punting that to #3444 I added a comment there? I think that issue is going to end up being a bunch of sub issues addressing individual failures, and one of those should be switching to xfail for everything.

Works for me, though an alternative that lets us use xfail and avoid the having the process hang would be adding something like this at the start of each of these failing tests:

def test_something(...):
    if platform.system() == "Windows":
        pytest.xfail("reason...")
    # The original body of the test function continues here

Went ahead and added xfail instead of skip!

[WillDaSilva]

Thanks for getting this done @visch! I'm glad we'll be able to make these Windows tests available in CI in time for me to run the Python 3.10 changes against them.

[visch]

Code Coverage stuff isn't quite accurate here (Or maybe I misunderstand) as we are testing these things but it's dependent on our github actions. We could dive into this as a part of this PR as well? (this is new as well!)

[edgarrmondragon]

Code Coverage stuff isn't quite accurate here (Or maybe I misunderstand) as we are testing these things but it's dependent on our github actions. We could dive into this as a part of this PR as well? (this is new as well!)

@visch could it be because we're only analyzing coverage for the results of testing in 3.9?

[visch]

Code Coverage stuff isn't quite accurate here (Or maybe I misunderstand) as we are testing these things but it's dependent on our github actions. We could dive into this as a part of this PR as well? (this is new as well!)

@visch could it be because we're only analyzing coverage for the results of testing in 3.9?

I don't know, I wouldn't think so but I have no idea.

I guess my question here is do we need to fix code coverage here or not?

[edgarrmondragon]

I guess my question here is do we need to fix code coverage here or not?

@visch not at all. It's currently only a guideline and not a strict requirement.

[WillDaSilva]

@visch The new tests failures are my bad: #6306

Sorry about that. I'll have it sorted out as soon as possible.