Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update WindowsAppSDK-RunHelixTests-Job.yml #3495

Merged
merged 46 commits into from
Mar 15, 2023

Conversation

alexlamtest
Copy link
Contributor

@alexlamtest alexlamtest commented Mar 10, 2023

Migrated MRT's MrmBaseUnitTests, MrmUnitTest, and MrtCoreUnpackagedTests test suites from Helix to the pipeline test environment, leveraging the recently introduced .testdef mechanism for onboarding them. There is then no longer a need for WindowsAppSDK-RunHelixTests-job.yml, hence it is deleted in this PR.

Those 3 test suites were also being directly run on the current build job agent via the vstest task. That looks redundant now in the general case, because in the pipeline environment all TAEF tests are being run on many OS images, instead of just one version of Server. Hence, WindowsAppSDK-BuildAndTestMRT-Steps.yml is updated to stop running the 3 MRT test suites, unless the code is called from the CI pipeline, because in this specific flow we currently don't run any tests later in the 1ES hosted pipeline environment.

This PR not only allows us to drop dependency on Helix (whose support is dwindling) from the Foundation repo, but also makes the pipeline more efficient, e.g., in a sample pipeline run in the past: https://dev.azure.com/microsoft/ProjectReunion/_build/results?buildId=64221252&view=results, the "TestInHelix Release_x64" job that runs the MRT tests took 25m 23s.
In an early pipeline run of this PR: https://dev.azure.com/microsoft/ProjectReunion/_build/results?buildId=66122428&view=results, the "PipelineTestsX64 rs_prerelease_x64fre" that runs the same MRT suites, plus a couple of other pre-existing test suites there, took 12m 59s, i.e., a speed up of ~50%.

CR: Michael, Will, Robert

How built:

  • Changes built fine when triggering the TransportPackage-Foundation-PR pipeline manually
  • PR validation

How Tested:

  • In a test pipeline run of this PR: https://dev.azure.com/microsoft/ProjectReunion/_build/results?buildId=66131963&view=results, MRT's MrmBaseUnitTests.dll, MrmUnitTest.dll, and MrtCoreUnpackagedTests.dll produced results below, respectively, as expected in the "PipelineTestsX64 rs_prerelease_x64fre" job:
    Summary: Total=268, Passed=268, Failed=0, Blocked=0, Not Run=0, Skipped=0
    Summary: Total=21, Passed=21, Failed=0, Blocked=0, Not Run=0, Skipped=0
    Summary: Total=27, Passed=27, Failed=0, Blocked=0, Not Run=0, Skipped=0

A microsoft employee must use /azp run to validate using the pipelines below.

WARNING:
Comments made by azure-pipelines bot maybe inaccurate.
Please see pipeline link to verify that the build is being ran.

For status checks on the develop branch, please use TransportPackage-Foundation-PR
(https://microsoft.visualstudio.com/ProjectReunion/_build?definitionId=81063&_a=summary)
and run the build against your PR branch with the default parameters.

For status checks on the main branch, please use microsoft.ProjectReunion
(https://dev.azure.com/ms/ProjectReunion/_build?definitionId=391&_a=summary)
and run the build against your PR branch with the default parameters.

@alexlamtest
Copy link
Contributor Author

/azp run

@azure-pipelines
Copy link

Azure Pipelines could not run because the pipeline triggers exclude this branch/path.

@alexlamtest
Copy link
Contributor Author

/azp run

@azure-pipelines
Copy link

Azure Pipelines could not run because the pipeline triggers exclude this branch/path.

@alexlamtest
Copy link
Contributor Author

/azp run

@azure-pipelines
Copy link

Azure Pipelines could not run because the pipeline triggers exclude this branch/path.

Copy link
Contributor

@codendone codendone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@alexlamtest alexlamtest enabled auto-merge (squash) March 15, 2023 21:06
@alexlamtest alexlamtest merged commit 46ef7f6 into develop Mar 15, 2023
@alexlamtest alexlamtest deleted the user/alexlam/43277347-dev-20230309 branch March 15, 2023 21:24
alexlamtest added a commit that referenced this pull request Mar 15, 2023
* Update WindowsAppSDK-RunHelixTests-Job.yml

* Update WindowsAppSDK-BuildFoundation-job.yml

* Add files via upload

* Update MrtCoreUnpackagedTests.csproj

* Add files via upload

* Update MrmUnitTest.vcxproj

* Add files via upload

* Update MrmBaseUnitTests.vcxproj

* Update WindowsAppSDK-BuildFoundation-job.yml

* Update WindowsAppSDK-RunHelixTests-Job.yml

* Update WindowsAppSDK-RunTestsInPipeline-Job.yml

* Update WindowsAppSDK-RunHelixTests-Job.yml

* Update WindowsAppSDK-BuildFoundation-job.yml

* Update MrmUnitTests.testdef

* Update MrtCoreUnpackagedTests.testdef

* Update MrmBaseUnitTests.testdef

* Update WindowsAppSDK-BuildFoundation-job.yml

* Update WindowsAppSDK-RunTestsInPipeline-Job.yml

* Update tsaoptions.json

* Update WindowsAppSDK-BuildAndTestMRT-Steps.yml

* Delete MrmUnitTests.testdef

* Add files via upload

* Update MrmUnitTest.vcxproj

* Update WindowsAppSDK-BuildAndTestMRT-Steps.yml

* Update WindowsAppSDK-BuildFoundation-job.yml

* Update WindowsAppSDK-BuildFoundation-job.yml

* Update WindowsAppSDK-RunTestsInPipeline-Job.yml

* Delete WindowsAppSDK-RunHelixTests-Job.yml

* Update tsaoptions.json

* Update ProjectReunion-CI.yml

* Delete RunTestsInHelix.proj

* Delete global.json

* Delete packages.config

* Delete WindowsAppSDK.Helix.Test.sln

* Delete Program.cs

* Delete WindowsAppSDK.Helix.Test.NetCore.csproj

* Delete TestAssembly.cs

* Delete TestMachineSetup.cmd

* Delete TestPass-EnsureMachineState.ps1

* Delete TestPass-OneTimeMachineSetup.ps1

* Delete TestPass-PostRun.ps1

* Delete TestPass-PreRun.ps1

* Delete WindowsAppSDK.Helix.TestCommon.csproj

* Update WindowsAppSDK-BuildAndTestMRT-Steps.yml
alexlamtest added a commit that referenced this pull request Mar 16, 2023
* Update WindowsAppSDK-RunHelixTests-Job.yml (#3495)

* Update WindowsAppSDK-RunHelixTests-Job.yml

* Update WindowsAppSDK-BuildFoundation-job.yml

* Add files via upload

* Update MrtCoreUnpackagedTests.csproj

* Add files via upload

* Update MrmUnitTest.vcxproj

* Add files via upload

* Update MrmBaseUnitTests.vcxproj

* Update WindowsAppSDK-BuildFoundation-job.yml

* Update WindowsAppSDK-RunHelixTests-Job.yml

* Update WindowsAppSDK-RunTestsInPipeline-Job.yml

* Update WindowsAppSDK-RunHelixTests-Job.yml

* Update WindowsAppSDK-BuildFoundation-job.yml

* Update MrmUnitTests.testdef

* Update MrtCoreUnpackagedTests.testdef

* Update MrmBaseUnitTests.testdef

* Update WindowsAppSDK-BuildFoundation-job.yml

* Update WindowsAppSDK-RunTestsInPipeline-Job.yml

* Update tsaoptions.json

* Update WindowsAppSDK-BuildAndTestMRT-Steps.yml

* Delete MrmUnitTests.testdef

* Add files via upload

* Update MrmUnitTest.vcxproj

* Update WindowsAppSDK-BuildAndTestMRT-Steps.yml

* Update WindowsAppSDK-BuildFoundation-job.yml

* Update WindowsAppSDK-BuildFoundation-job.yml

* Update WindowsAppSDK-RunTestsInPipeline-Job.yml

* Delete WindowsAppSDK-RunHelixTests-Job.yml

* Update tsaoptions.json

* Update ProjectReunion-CI.yml

* Delete RunTestsInHelix.proj

* Delete global.json

* Delete packages.config

* Delete WindowsAppSDK.Helix.Test.sln

* Delete Program.cs

* Delete WindowsAppSDK.Helix.Test.NetCore.csproj

* Delete TestAssembly.cs

* Delete TestMachineSetup.cmd

* Delete TestPass-EnsureMachineState.ps1

* Delete TestPass-OneTimeMachineSetup.ps1

* Delete TestPass-PostRun.ps1

* Delete TestPass-PreRun.ps1

* Delete WindowsAppSDK.Helix.TestCommon.csproj

* Update WindowsAppSDK-BuildAndTestMRT-Steps.yml

* 43277347: Manaully apply some changes in the CP to a source file that diverged between the 1.3-stable and Develop branches.

---------

Co-authored-by: Alex Lam (APPX) <alexlam@ntdev.microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants