VSTest@2 not compatible with Visual Studio 2022

[nicolasnoble]

Required Information

Type: Bug

Enter Task Name: VSTest@2

https://github.com/microsoft/azure-pipelines-tasks/tree/master/Tasks/VsTestV2

Environment

• Server - Azure Pipelines
• Agent - Hosted

Using the windows-2022 image, which has Visual Studio 2022 installed.

Specific run:
https://dev.azure.com/grumpycoders/pcsx-redux/_build/results?buildId=6306&view=logs&j=12f1170f-54f2-53f3-20dd-22fc7dff55f9&t=7bfb7e0f-549d-5929-c603-6e8046613b9e&l=24

Issue Description

Task logs

Starting: VSTest
==============================================================================
Task         : Visual Studio Test
Description  : Run unit and functional tests (Selenium, Appium, Coded UI test, etc.) using the Visual Studio Test (VsTest) runner. Test frameworks that have a Visual Studio test adapter such as MsTest, xUnit, NUnit, Chutzpah (for JavaScript tests using QUnit, Mocha and Jasmine), etc. can be run. Tests can be distributed on multiple agents using this task (version 2).
Version      : 2.170.1
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/test/vstest
==============================================================================
##[warning]This task uses Node 6 execution handler, which will be deprecated soon. If you are the developer of the task - please consider the migration guideline to Node 10 handler - https://aka.ms/migrateTaskNode10. If you are the user - feel free to reach out to the owners of this task to proceed on migration.
SystemVssConnection exists true
SystemVssConnection exists true
Running tests using vstest.console.exe runner.
======================================================
Test selector : Test assemblies
Test filter criteria : null
Search folder : D:\a\1\s
Action when minimum tests threshold not met : donothing
Minimum tests expected to be run: 0
VisualStudio version selected for test execution : latest
Attempting to find vstest.console from a visual studio installation with version [16.0,17.0).
Attempting to find vstest.console from a visual studio build tools installation with version [16.0,17.0).
Attempting to find vstest.console from a visual studio installation with version [15.0,16.0).
Attempting to find vstest.console from a visual studio build tools installation with version [15.0,16.0).
##[error]Error: Visual Studio 2015 is not found. Try again with a version that exists on your build agent machine.
Finishing: VSTest

Troubleshooting

Error logs

Attempting to find vstest.console from a visual studio installation with version [16.0,17.0).
Attempting to find vstest.console from a visual studio build tools installation with version [16.0,17.0).
Attempting to find vstest.console from a visual studio installation with version [15.0,16.0).
Attempting to find vstest.console from a visual studio build tools installation with version [15.0,16.0).
##[error]Error: Visual Studio 2015 is not found. Try again with a version that exists on your build agent machine.

[phanikmmsft]

@nadesu can you please help assign to the right owner?

[nicolasnoble]

FWIW, I've tried forcing version number of 17.0 without success. The existing code has some remnants of testing for [17.0,18.0), but there's further codepaths down that squashes the attempt.

Visual Studio 2022 has been officially out for a little while now, and this means we can't test its builds on Azure Pipelines properly that depend on the visual studio test runner.

[nicolasnoble]

Any update on this? It looks like we still can't test Visual Studio 2022 on Azure Pipelines...

[nicolasnoble]

Looks like this was in fact addressed by #15400. Closing this one.

[DigCamara]

Seems to me that this bug should not have been closed. In one of the changed files the validation of the version was not modified (line 246).