Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When ClassInitialize fails only one test fails, others executed regardless #672

Closed
cklutz opened this issue Jan 7, 2020 · 5 comments
Closed
Assignees

Comments

@cklutz
Copy link

cklutz commented Jan 7, 2020

Description

This is based on this StackOverflow question: https://stackoverflow.com/q/59624616/21567

When you have a test class with a ClassInitialize-Method and that method fails. Only one of the actual test methods is reported as failed. The other methods seem to execute regardless.

Steps to reproduce

  • Create a .NET Core UnitTest project in Visual Studio 2019
  • Add the following test class
    [TestClass]
    public class UnitTest1
    {
        [ClassInitialize]
        public static void ClassInit(TestContext tc)
        {
            throw new Exception("DIE IN INITIALIZE");
        }

        [TestMethod] public void TestMethod1() { Console.WriteLine(nameof(TestMethod1)); }
        [TestMethod] public void TestMethod2() { Console.WriteLine(nameof(TestMethod2)); }
        [TestMethod] public void TestMethod3() { Console.WriteLine(nameof(TestMethod3)); }
    }

Project File:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>netcoreapp3.1</TargetFramework>
    <IsPackable>false</IsPackable>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.2.0" />
    <PackageReference Include="MSTest.TestAdapter" Version="2.0.0" />
    <PackageReference Include="MSTest.TestFramework" Version="2.0.0" />
    <PackageReference Include="coverlet.collector" Version="1.0.1" />
  </ItemGroup>
</Project>
  • run dotnet test

Expected behavior

All test methods are "failed" (i.e. not run, because the initializer method failed). That was also the behaviour with MSTest 1.3.0 / full Framework.

Actual behavior

Only the first test (here TestMethod1) is failed with the correct error message (that the initializer failed). Other test methods run anyway (output Console.WriteLine messages). Messages will possibly only be visible from VS Test Explorer because of microsoft/vstest#799.

Environment

  • Windows 10
  • Visual Studio 2019 16.3
  • MSTest.TestAdapter 2.0.0
  • MSTest.TestFramework 2.0.0
@thedefside
Copy link

I noticed the same issue today.
VS 2019
.NET Framework 4.7.2
MSTest 2.0.0

@abhijitpai
Copy link

Seems like this behavior may have changed in #577 inadvertently.

@ravarnamsft
Copy link

We are seeing this behavior as well for .Net Core. This is making debugging very hard as a lot of tests fail later on because the setup was incomplete. We have to go and try finding which test kicked off the class initialize and see its logs. @parrainc and @AbhitejJohn is there a workaround or a fix planned for this?

@AbhitejJohn
Copy link
Contributor

Tagging @Haplois to help.

@Evangelink
Copy link
Member

Hey folks! I am reviewing this issue and managed to reproduce with the version indicated but it seems that the issue was fixed with 2.2.0 and later versions.

I will move forward and close this issue as fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants