Stop building console runner with .NET 2.0

[CharliePoole]

@rprouse @jnm2 @mikkelbu @OsirisTerje @nunit/engine-team @nunit/framework-team

I plan to stop using .NET 2.0 for the console runner and the engine proper (nunit.engine.dll).

This needs discussions on three points... (1) when to do it (2) what target should replace it and (3) what should we keep building in .NET 2.0 (if anything) in order to support tests.

---

When

Strictly speaking, this is a breaking change. Folks who want to run tests on a machine, which only has .NET 2.0 installed will no longer be able to do that. OTOH, it's hard to imagine that there are many such people around. Personally, I would loosen the rule of only doing breaking changes in major releases and include it in 3.16. OTOH, I don't feel strongly about the timing, especially if somebody can uncover a significant group of users who would be negatively affected. In that case it would become a 4.0 change.

Our decision on this point will affect what we do with respect to the other two points.

New Target

I would propose using 4.6.2 or higher. Since Microsoft support of these versions is tied to the underlying OS, we may want to have some discussion around this. Note that this issue is only about replacing .NET 2.0, so we aren't discussiong .NET Core versions. Creating a full-featured .NET Core console runner is a separate issue.

Running .NET 2.0 tests

One option is to say that we will only support .NET 2.0 (or 3.x) tests running under .NET 4.x. I think this might be a negative impact for some group of users, although probably relatively small. For 3.16, I would err on the side of caution, keeping a .NET 2.0 build of the nunit.engine.api and nunit.engine.core assemblies as well as the .NET 2.0 agents.

For .NET 4.0, I think that the 2.0 agent should be kept available as an extension (pluggable agent) and not delivered by default. Note that such an agent already exists. How to handle nunit.engine.api and nunit.engine.core in 4.0 will require some design work.

---

I'd appreciate your comments so I can better flesh out the contents of the 3.16 release!

[jnm2]

To me, forcing test runs on CLR v4 instead of CLR v2 (which is the runtime engine that actually executes .NET Framework 3.5 apps) seems like a major change. Many people won't notice, but some will. There are arcane edge cases such as numeric precision or System.Drawing behaving differently on v4 which can be confusing when your tests start failing.

Further points of reference:

• NUnit Framework supports 3.5 but dropped support for 2.0 in v3.12 in May 2019. There's been discussion of dropping support for 3.5 and older 4.x versions of .NET Framework for v4 of NUnit Framework, but as far as I remember, nothing is decided.

• dotnet test (the VSTest runner) runs all .NET Framework 3.5 and older tests against .NET Framework 4.0.

• .NET Framework 3.5 SP1 is officially supported by Microsoft until Jan 2029. https://learn.microsoft.com/en-us/lifecycle/products/microsoft-net-framework

• Other than 3.5, .NET Framework 4.6.2 is the oldest version supported by Microsoft.

[rprouse]

Extended support for .NET 2.0 ended in 2016 so I don't consider this a breaking change, but I've always been against running out of support software for security reasons. .NET 3.5 makes sense as a target because of its long-term support. On the other hand, targeting older frameworks might discourage people from contributing to the project.

[CharliePoole]

@jnm2 I did list running CLR 2 tests on CLR 4 as an option, but one I don't like. I prefer to keep the ability to run tests under CLR 2 available for whomever needs it. But my main proposal is that we build the console runner and engine for some CLR 4 target.

The result would be that anyone wanting to run tests under CLR 2 could do so, but they would also need to have some CLR 4 runtime installed for the runner itself to use.

So the users affected would be an even smaller group: those who want to run tests on a machine with only CLR 2 installed. They would have to use NUnitLite to run tests. It would also impact anyone wanting to run CLR 2 tests in process - that would be impossible. However, even though it's a small group, I agree that it is a breaking change and if you feel strongly I can postpone it till 4.0.

Ultimately, I imagine you will need .NET 7.0 or 8.0 to run the console, even though we could still support older runtimes on the agent side of things.

[jnm2]

@CharliePoole Ah, thanks. I am a fan of adding a .NET Framework 4.6.2 target then as a minor update.

Those who want to run tests on a machine with only CLR 2 installed."

For reference, such machines are Windows 7 and older, or Windows Server 2008 and older. Everything newer comes with .NET Framework 4.x installed as a necessary OS component. The oldest versions of Windows in support are 8.1 (for a few more months) and Server 2012.

[jnm2]

Updated my last comment because I was missing the significance of icons in the reference link.

[CharliePoole]

@jnm2 There are also those folks who use specially configured machines or devices as, for example, in certain medical applications I have worked on. Even so, new development would necessarily be using supported runtimes.

[CharliePoole]

Based on the comments, here's my plan:

For 3.16

• Target the console runner itself and the engine assembly to 4.6.2. (this issue)
• Retain the 2.0 and 4.0 agents, as well as the 2.0 and 4.0 nunit.engine.core and nunit.engine.api builds.

For 4.0

• Eliminate 2.0 as a built-in agent, but support the 2.0 pluggable agent.
• Consider replacing our 4.0 targets with 4.6.2 or adding a 4.6.2 target.

I'm adding this to the 3.16 milestone. Comments are still welcome, of course.

[CharliePoole]

In addition, since we require at least 4.6.2 to be installed, I'll update the agent as well as test data files like mock-assembly from net40 to net462.

[CharliePoole]

This issue has been resolved in version 3.16.0

The release is available on:
GitHub.
NuGet packages are also available NuGet.org and
Chocolatey Packages may be found at Chocolatey.org