NUnit Console cannot run tests with TargetFramework "net5.0" #858
Comments
|
The output displayed indicates that the console runner itself targets .net core 3.1. So too does the engine used by the runner. However, it says nothing about the process in which your tests are running. When we build the runner, we actually run tests under .NET 5.0, if it is installed. However, it's possible that the beta-1 release is still not able to handle that. I suggest trying the latest @ChrisMaddock you probably have a more complete answer for this than I do. |
|
I have updated the console with latest version in myGet 3.12.0-dev-04990, but I received the same error: |
|
AND in myGet page says in "Target frameworks" section ".NETCoreApp 3.1: 3.1.0.0", no mention to "net5.0": |
|
Finally (don't take me wrong, I'm not being annoying, just helping on debugging). Anyway, if it can help, this is the content of the auto generated project file, targeting .net5: Yeah, I noticed the NUnit 3.12.0 version. I updated to 3.13, and NUnit3TestAdapter to 4.0.0-beta.1 but it does not work too. And guess what happens when I change the TargetFramework in project file to netcoreapp3.1? |
|
If @ChrisMaddock, who wrote the .NET Core runner, jumps in here, you can just ignore me, since I'm focused on other areas. But here's what seems to be happening.
Hope that's clear. I'm about 90% sure it's right 😕 because what you are doing is not exactly what we do in our own tests. We don't yet have a 5.0 build, so we are running 3.1 tests under 5.0. IOW, YMMV. 😄 |
|
Hey @CharliePoole, it works now! 😄 My problem was, I'm calling directly nunit3-console.exe or nunit (the global dotnet cli tool) in command line. With your tip about
and the result is: It correctly detected net5.0... I think I can use it like this, but is this (or will be) the correct way to execute it from now on? |
|
As you saw, there are currently two runners, one for .NET Framework and another for .NET Core. I initially thought you might be using the wrong one, but you talked about running the dotnet command, so I figured you were not. Having two runners is probably not a long term solution, because some folks will want to run projects that include both framework and core test assemblies. However, it's where we are right now. If you want to try using a GUI, I invite you to look at my TestCentric GUI, which takes the approach of a single runner with multiple agents. Unfortunately, I haven't yet published a .NET 5.0 agent. Both projects are looking at decoupling agents from the engine in the future, so they become pluggable. |
|
This TestCentric GUI is amazing, but "unfortunatelly" 😄 you help me on fix my issue so my tests will be compiled and executed in .Net 5 now, and then this tool is obsolete for me. But really I will wait (or maybe help) on building your .Net 5 agent. |
|
Cute! Actually, creating a .net 5 agent is relatively trivial and has potential for wide benefits. I'll add an issue. |
|
FYI... the GUI now has a .NET 5.0 agent. |
|
I would like to point out that the suggested --fx-version parameter is only usable if you do not use the console runner directly. In our environment, we use unit tests via dotCover to measure code coverage too. So if I have
Then it works fine, I have a nice test output and a test coverage output as well because "dotnet test" will use the appropriate runtime, as it is shown during the execution: "Test run for c:\xxx\CarShop.Tests\bin\Debug\net5.0\CarShop.Tests.dll (.NETCoreApp,Version=v5.0)" However, if I want to use the console runner directly, it does not work:
In the second case, I cannot set the runtime version, nunit3-console will say: "Invalid argument: --fx-version"... As a result, I have failure: In september we managed to switch to dotnet core with the test runners as you released the "NUnit Console and Engine 3.12.0 Beta 1" on 1/Aug. First of all, I'd like to say I am very thankful for that :) But now we'd like to switch to .net5.0 in February, but the standalone console runner does not work as expected... Using "dotnet test" is a good workaround. And also, the .net5 tests are executed nicely with the latest TestCentricGui ZIP package ... So a question to @CharliePoole : what is the future plan with the runners?
(+1: btw, are there any advantages for the console runner over "dotnet test"? I see the XML report format is different, but is there any differences functionality-wise???) ps> Thank you for your project!!! |
|
@anoftc First off, I can only try to answer questions about what NUnit3 Console now does. I'm not the guy to tell you what the plans are. As far as your two command examples go, they are not comparable. The first doesn't run the console runner at all while the second does. In the second command, you have to pass the WRT your questions...
To your BTW question: Mainly, dotnet test relies on vstest, which has a general picture of tests that doesn't match NUnit's so well. For example, it doesn't know about warnings, inconclusive results, theories, explicit tests... so you have to know not to use any of those features when you use that runner. Not a criticism of the product, just a fact of life whenever you go through an adapter layer between two pieces of software, especially a generalized layer. Charlie |
|
@anoftc, this explanation from @CharliePoole is exactly what is happening with me.
It works but it has disadvantages too. I successfully deployed to a "mcr.microsoft.com/dotnet/aspnet:5.0" container. Yesterday the code stopped to work in my computer. Guess what? I updated the Visual Studio version and it updated my .Net5 to 5.0.2, and how this code expects 5.0.1 it does not works anymore in my PC. And the worse part, Or maybe nunit3-console.exe will be updated to deal with all this situation, what comes first 😄. |
|
I ran into this problem when updating our large WPF application to net5.0. Is this related to our test project targeting net5.0-windows instead of just plain net5.0? |
|
@rprouse To me it seems like this is a blocking issue for using the nunit console runner with net5/net6. |
|
I think we need @ChrisMaddock's opinion on this since he did most of the work on the .NET Core version of the runner. I admit that the last minute changes in .NET 5 caught us off guard. Until we get this fixed in the console runner, have you tried adding the NUnit VS Test Adapter to your projects and running using VSTest or |
|
@oysteinkrog You can run using the windows variant of .NET 5 by navigating to |
|
@nmuench Thank you, that does indeed work :) |
|
This is straight-forward but slightly time-consuming because there are lots of moving parts. Questions for @nunit/engine-team ...
My own answers...
|
|
OK... I did a spike. TLDR; I'm not going to try this as part of the 3.13 release. The reason I thought it would be fairly straightforward is that I already run a .NET 5.0 agent in the testcentric fork of the engine. However, in that agent, there are other significant changes underlying the change, which are not present in the NUnit engine. I think we should talk about this in planning for future development, after the 3.13 release. |
|
Answers to the questions
|
|
NOTE to self... I need to detail just what the "underlying changes" are required. |
|
Both the netfx and net core runners are discussed here but that's primarily due to my confusion. The original issue pertained to the net core runner, so I'm labeling it accordingly. Adding a .NET 5.0 agent to NUnit.ConsoleRunner.NetCore would be one BIG enhancement, since the .Net Core runner does not currently support agents at all! We could consider adding a .NET 5.0 build, however. If I understand correctly, a .NET Core tool can have multiple builds of the exe included. @nunit/engine-team Let's discuss this. |
CharliePoole
added
Needs Discussion
Enhancement
NetCoreRunner
The usual disclaimer :): I don't know anything about dot net core tools, but if we should add a .NET 5 target, then we should also add a .NET 6 target. I tried to google for information about multiple builds of the exe, but I'm didn't find anything useful. |
|
I wonder if adding net5.0 and net6.0 builds of the runner might cause all netcoreapp3.1 tests to start running under .NET 5 or 6, whatever is available on the machine. Would there be any way for a user of the tool to control this? If this isn't the path forward, then agents seem to be. Maybe instead of making the NetCore package be able to run agents, we make the |
|
@jnm2 I sent you a separate note about a problem I'm having with the .NET 5.0 Agent in the Azure Linux job. But the good news is that it works everywhere else. This agent approach is what I was pushing for originally. I felt that the NetCore runner wasn't the right way to go. However, since some people can only run under .NET Core it appears we need both approaches. |
|
Comments on this issue are pretty cluttered, in large part because I tried to help without understanding that the current NETCORE console runner doesn't use agents at all! Everything is run in process. Now I'm involved with both runners - NETFX and NETCORE - so I'd like to start over and try to eliminate the confusion... including any that my earlier comments may have caused. So I'm closing this issue and creating a new one to replace it. Before doing that, I'll summarize where we are on this issue...
@nunit/engine-team Please correct me if you think any of the above points is wrong!!! |
If it matters, I'm using:
When I compile my tests with TargetFramework "netcoreapp3.1", the console works fine.
When I compile my tests with TargetFramework "net5.0", the console cannot execute them.
This is the output:
Looking the console output I see the runner is using "Runtime: .NET Core 3.1.10". I cannot find an option in command line to change this (maybe it is not detecting correctly yet?). I tested adding the parameter
--framework=net5.0, but it outputs:The framework= option is not available on this platform.My questions:
The text was updated successfully, but these errors were encountered: