Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

'dotnet test' broken after upgrading .NET SDK from 7.0.302 to 8.0.100 #38473

Closed
AArnott opened this issue Feb 1, 2024 · 8 comments
Closed

'dotnet test' broken after upgrading .NET SDK from 7.0.302 to 8.0.100 #38473

AArnott opened this issue Feb 1, 2024 · 8 comments
Labels
Area-DotNet Test untriaged Request triage from a team member

Comments

@AArnott
Copy link
Contributor

AArnott commented Feb 1, 2024

Describe the bug

dotnet test fails with the following error after upgrading to the .NET 8 SDK:

  An assembly specified in the application dependencies manifest (testhost.deps.json) was not found:
    package: 'NuGet.Frameworks', version: '6.5.0'
    path: 'lib/netstandard2.0/NuGet.Frameworks.dll'
. Please check the diagnostic logs for more information.

To Reproduce

git clone https://devdiv@dev.azure.com/devdiv/DevDiv/_git/VS.CloudCache
cd VS.CloudCache
git checkout f89f466
cd test\Microsoft.VisualStudio.Cache.Tests
dotnet test

It works.

Now change global.json at the repo root to consume 8.0.100 of the .NET SDK:

-    "version": "7.0.302",
+    "version": "8.0.100",

Run dotnet test again, and it'll fail:


  Determining projects to restore...
  Restored C:\Users\andarno\source\repos\VS.CloudCache\src\Microsoft.VisualStudio.Cache\Microsoft.VisualStudio.Cache.csproj (in 2.25 sec).
  Restored C:\Users\andarno\source\repos\VS.CloudCache\test\Microsoft.VisualStudio.Cache.Tests\Microsoft.VisualStudio.Cache.Tests.csproj (in 4.23 sec).
  Microsoft.VisualStudio.Cache -> C:\Users\andarno\source\repos\VS.CloudCache\bin\Microsoft.VisualStudio.Cache\Debug\net6.0-windows10.0.19041\Microsoft.VisualStudio.Cache.dll
  Microsoft.VisualStudio.Cache.Tests -> C:\Users\andarno\source\repos\VS.CloudCache\bin\Microsoft.VisualStudio.Cache.Tests\x64\Debug\net6.0-windows10.0.19041\win-x64\Microsoft.VisualStudio.Cache.Tests.dll
Test run for C:\Users\andarno\source\repos\VS.CloudCache\bin\Microsoft.VisualStudio.Cache.Tests\x64\Debug\net6.0-windows10.0.19041\win-x64\Microsoft.VisualStudio.Cache.Tests.dll (.NETCoreApp,Version=v6.0)
Microsoft (R) Test Execution Command Line Tool Version 17.8.0 (x64)
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
Testhost process for source(s) 'C:\Users\andarno\source\repos\VS.CloudCache\bin\Microsoft.VisualStudio.Cache.Tests\x64\Debug\net6.0-windows10.0.19041\win-x64\Microsoft.VisualStudio.Cache.Tests.dll' exited with error: Error:
  An assembly specified in the application dependencies manifest (testhost.deps.json) was not found:
    package: 'NuGet.Frameworks', version: '6.5.0'
    path: 'lib/netstandard2.0/NuGet.Frameworks.dll'
. Please check the diagnostic logs for more information.

Test Run Aborted.

Build FAILED.
    0 Warning(s)
    0 Error(s)

I have no idea why it would fail like this, considering this test has no NuGet.Frameworks dependency at all. And I can't find a testhost.deps.json file in the output directory either.

dotnet --info output in details:

dotnet --info .NET SDK: Version: 8.0.101 Commit: 6eceda187b Workload version: 8.0.100-manifests.077a4f18

Runtime Environment:
OS Name: Windows
OS Version: 10.0.22621
OS Platform: Windows
RID: win-x64
Base Path: C:\Program Files\dotnet\sdk\8.0.101\

.NET workloads installed:
Workload version: 8.0.100-manifests.077a4f18
[aspire]
Installation Source: VS 17.10.34601.30, VS 17.10.34531.228
Manifest Version: 8.0.0-preview.1.23557.2/8.0.100
Manifest Path: C:\Program Files\dotnet\sdk-manifests\8.0.100\microsoft.net.sdk.aspire\8.0.0-preview.1.23557.2\WorkloadManifest.json
Install Type: Msi

Host:
Version: 8.0.1
Architecture: x64
Commit: bf5e279d92

.NET SDKs installed:
6.0.418 [C:\Program Files\dotnet\sdk]
7.0.115 [C:\Program Files\dotnet\sdk]
7.0.203 [C:\Program Files\dotnet\sdk]
7.0.312 [C:\Program Files\dotnet\sdk]
7.0.405 [C:\Program Files\dotnet\sdk]
8.0.101 [C:\Program Files\dotnet\sdk]
8.0.200-preview.24060.48 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
Microsoft.AspNetCore.App 6.0.25 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 6.0.26 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 7.0.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 7.0.14 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 7.0.15 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 8.0.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 8.0.1 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 6.0.25 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.26 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 7.0.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 7.0.14 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 7.0.15 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 8.0.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 8.0.1 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.WindowsDesktop.App 6.0.25 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.26 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 7.0.5 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 7.0.14 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 7.0.15 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 8.0.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 8.0.1 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Other architectures found:
x86 [C:\Program Files (x86)\dotnet]
registered at [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\x86\InstallLocation]

Environment variables:
Not set

global.json file:
C:\Users\andarno\source\repos\VS.CloudCache\global.json

Learn more:
https://aka.ms/dotnet/info

Download .NET:
https://aka.ms/dotnet/download

@dotnet-issue-labeler dotnet-issue-labeler bot added Area-DotNet Test untriaged Request triage from a team member labels Feb 1, 2024
@vitek-karas
Copy link
Member

@nohwnd Could you please redirect as appropriate?

@nohwnd
Copy link
Member

nohwnd commented Feb 2, 2024

Nuget.Frameworks is a dependency of testhost. But your project is not referencing Microsoft.NET.SDK so it falls back to using testhost that we ship in dotnet folder.

exec --runtimeconfig "S:\c\VS.CloudCache\bin\Microsoft.VisualStudio.Cache.Tests\x64\Debug\net6.0-windows10.0.19041\win-x64\Microsoft.VisualStudio.Cache.Tests.runtimeconfig.json" --depsfile "S:\c\VS.CloudCache\bin\Microsoft.VisualStudio.Cache.Tests\x64\Debug\net6.0-windows10.0.19041\win-x64\Microsoft.VisualStudio.Cache.Tests.deps.json" --additional-deps "C:\Program Files\dotnet\sdk\8.0.101\testhost.deps.json" --additionalprobingpath "C:\Program Files\dotnet\sdk\8.0.101" "C:\Program Files\dotnet\sdk\8.0.101\testhost.dll" --port 18553 --endpoint 127.0.0.1:018553 --role client --parentprocessid 51516 --diag "S:\c\VS.CloudCache\test\Microsoft.VisualStudio.Cache.Tests\logs\dotnet_51516.host.24-02-02_13-07-20_60928_5.diag" --tracelevel 4 --telemetryoptedin false'

dotnet sdk is replacing our version of nuget.frameworks so I guess this fallback is broken again. Posted a PR to the repo, so it does not hit this fallback.

https://dev.azure.com/devdiv/DevDiv/_git/VS.CloudCache/pullrequest/526038

@nohwnd
Copy link
Member

nohwnd commented Feb 2, 2024

Will fix on vstest microsoft/vstest#4875 . And probably add test in here.

Closing because projects should reference Microsoft.NET.Test.SDK if possible. Which this project can and it fixes it.

@nohwnd nohwnd closed this as completed Feb 2, 2024
@AArnott
Copy link
Contributor Author

AArnott commented Feb 2, 2024

Thanks, @nohwnd!

@robertmclaws
Copy link

robertmclaws commented Mar 27, 2024

Referencing the Microsoft.NET.Test.Sdk NuGet package did NOT fix this issue for dotnet test commands. They are still failing in build environments.

Still experiencing the issue on .NET 8.0.203 when referencing the following packages in test projects:

<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.*-*" />
<PackageReference Include="Microsoft.TestPlatform" Version="17.*-*" />

@PauloJoseBR
Copy link

I got the same problem @robertmclaws. Have you found a solution?

@duongphuhiep
Copy link

up vote

@nohwnd
Copy link
Member

nohwnd commented Apr 10, 2024

The technical issue was the same for Robert. He had a project that was marked as a test project, but it didn’t have the Microsoft.NET.Test.SDK installed.

In their case they set true build property on their utility project. This property tells vstest to run tests from it.

Solution for them was to not set that property.

DavidMatthew1987 added a commit to DavidMatthew1987/wyoming that referenced this issue Jun 20, 2024
add necessary packages to run dotnet test
I mean the test still fails but that is a problem with my city code :yay:

wyoming-4730637:~/wyoming{main}$ dotnet test
  Determining projects to restore...
  All projects are up-to-date for restore.
  myapp -> /home/user/wyoming/bin/Debug/net8.0/myapp.dll
Test run for /home/user/wyoming/bin/Debug/net8.0/myapp.dll (.NETCoreApp,Version=v8.0)
Microsoft (R) Test Execution Command Line Tool Version 17.8.0 (x64)
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
Testhost process for source(s) '/home/user/wyoming/bin/Debug/net8.0/myapp.dll' exited with error: Error:
  An assembly specified in the application dependencies manifest (testhost.deps.json) was not found:
    package: 'Newtonsoft.Json', version: '13.0.1'
    path: 'lib/netstandard2.0/Newtonsoft.Json.dll'
. Please check the diagnostic logs for more information.

Test Run Aborted.
wyoming-4730637:~/wyoming{main}$ dotnet test
  Determining projects to restore...
  Restored /home/user/wyoming/myapp.csproj (in 1.31 sec).
  myapp -> /home/user/wyoming/bin/Debug/net8.0/myapp.dll
Test run for /home/user/wyoming/bin/Debug/net8.0/myapp.dll (.NETCoreApp,Version=v8.0)
Microsoft (R) Test Execution Command Line Tool Version 17.8.0 (x64)
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
Testhost process for source(s) '/home/user/wyoming/bin/Debug/net8.0/myapp.dll' exited with error: Error:
  An assembly specified in the application dependencies manifest (testhost.deps.json) was not found:
    package: 'NuGet.Frameworks', version: '6.5.0'
    path: 'lib/netstandard2.0/NuGet.Frameworks.dll'
. Please check the diagnostic logs for more information.

Test Run Aborted.
wyoming-4730637:~/wyoming{main}$ dotnet test
  Determining projects to restore...
  All projects are up-to-date for restore.
/home/user/.nuget/packages/microsoft.net.test.sdk/17.10.0/build/netcoreapp3.1/Microsoft.NET.Test.Sdk.Program.cs(4,41): warning CS7022: The entry point of the program is global code; ignoring 'AutoGeneratedProgram.Main(string[])' entry point. [/home/user/wyoming/myapp.csproj]
  myapp -> /home/user/wyoming/bin/Debug/net8.0/myapp.dll
Test run for /home/user/wyoming/bin/Debug/net8.0/myapp.dll (.NETCoreApp,Version=v8.0)
Microsoft (R) Test Execution Command Line Tool Version 17.8.0 (x64)
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
[xUnit.net 00:00:00.75]     AnalyticsWebApi.AnalyticsRepositoryTests.SaveAnalyticsData_ValidData_ReturnsSuccess [FAIL]
  Failed AnalyticsWebApi.AnalyticsRepositoryTests.SaveAnalyticsData_ValidData_ReturnsSuccess [254 ms]
  Error Message:
   NSubstitute.Exceptions.RedundantArgumentMatcherException : Some argument specifications (e.g. Arg.Is, Arg.Any) were left over after the last call.

This is often caused by using an argument spec with a call to a member NSubstitute does not handle (such as a non-virtual member or a call to an instance which is not a substitute), or for a purpose other than specifying a call (such as using an arg spec as a return value). For example:

    var sub = Substitute.For<SomeClass>();
    var realType = new MyRealType(sub);
    // INCORRECT, arg spec used on realType, not a substitute:
    realType.SomeMethod(Arg.Any<int>()).Returns(2);
    // INCORRECT, arg spec used as a return value, not to specify a call:
    sub.VirtualMethod(2).Returns(Arg.Any<int>());
    // INCORRECT, arg spec used with a non-virtual method:
    sub.NonVirtualMethod(Arg.Any<int>()).Returns(2);
    // CORRECT, arg spec used to specify virtual call on a substitute:
    sub.VirtualMethod(Arg.Any<int>()).Returns(2);

To fix this make sure you only use argument specifications with calls to substitutes. If your substitute is a class, make sure the member is virtual.

Another possible cause is that the argument spec type does not match the actual argument type, but code compiles due to an implicit cast. For example, Arg.Any<int>() was used, but Arg.Any<double>() was required.

NOTE: the cause of this exception can be in a previously executed test. Use the diagnostics below to see the types of any redundant arg specs, then work out where they are being created.

Diagnostic information:

Remaining (non-bound) argument specifications:
    any Object

All argument specifications:
    any String
    any Object

  Stack Trace:
     at NSubstitute.Core.Arguments.ArgumentSpecificationsFactory.Create(IList`1 argumentSpecs, Object[] arguments, IParameterInfo[] parameterInfos, MethodInfo methodInfo, MatchArgs matchArgs)
   at NSubstitute.Core.CallSpecificationFactory.CreateFrom(ICall call, MatchArgs matchArgs)
   at NSubstitute.Routing.Handlers.RecordCallSpecificationHandler.Handle(ICall call)
   at NSubstitute.Routing.Route.Handle(ICall call)
   at NSubstitute.Core.CallRouter.Route(ICall call)
   at NSubstitute.Proxies.CastleDynamicProxy.CastleForwardingInterceptor.Intercept(IInvocation invocation)
   at Castle.DynamicProxy.AbstractInvocation.Proceed()
   at NSubstitute.Proxies.CastleDynamicProxy.ProxyIdInterceptor.Intercept(IInvocation invocation)
   at Castle.DynamicProxy.AbstractInvocation.Proceed()
   at Castle.Proxies.ObjectProxy_1.set_CommandText(String value)
   at Dapper.CommandDefinition.SetupCommand(IDbConnection cnn, Action`2 paramReader) in /_/Dapper/CommandDefinition.cs:line 134
   at Dapper.SqlMapper.TrySetupAsyncCommand(CommandDefinition command, IDbConnection cnn, Action`2 paramReader) in /_/Dapper/SqlMapper.Async.cs:line 412
   at Dapper.SqlMapper.ExecuteImplAsync(IDbConnection cnn, CommandDefinition command, Object param) in /_/Dapper/SqlMapper.Async.cs:line 658
   at NSubstitute.SubstituteExtensions.ReThrowOnNSubstituteFault[T](Task`1 task)
   at NSubstitute.SubstituteExtensions.Returns[T](Task`1 value, T returnThis, T[] returnThese)
   at AnalyticsWebApi.AnalyticsRepositoryTests.SaveAnalyticsData_ValidData_ReturnsSuccess() in /home/user/wyoming/Tests/AnalyticsRepositoryTests.cs:line 24
--- End of stack trace from previous location ---

Failed!  - Failed:     1, Passed:     0, Skipped:     0, Total:     1, Duration: < 1 ms - myapp.dll (net8.0)
wyoming-4730637:~/wyoming{main}$

see dotnet/sdk#38473 for a fuller treatment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area-DotNet Test untriaged Request triage from a team member
Projects
None yet
Development

No branches or pull requests

6 participants