Add option for building a test exe as single file #42972

agocke · 2020-10-02T06:44:02Z

This is a work-in-progress PR, I haven't yet figured out how the xharness runners actually report test failures, since it doesn't look like they write the xunit results anywhere.

There's also the simpler matter of the exes not actually passing the tests...

ViktorHofer

This design will need to change as with #39923 the xunit.console targets and props files and the RunTests.cmd/sh generation for desktop configuration will go away. We should really get #39923 as that's part of our crash dump working group.

I'm marking this as request changes meanwhile to avoid an accidental merge.

marek-safar · 2020-10-02T15:40:35Z

/cc @akoeplinger

src/libraries/Common/tests/SingleFileTestRunner/SingleFileTestRunner.cs

src/libraries/System.Reflection/tests/System.Reflection.Tests.csproj

agocke · 2020-10-02T18:21:22Z

@ViktorHofer Haven't seen that PR before, I'll try to integrate.

I'd also like to treat this as progressive enhancement -- enable one project at at time. Idea on how best to do that? Is there a central list of the test projects which I could condition on PublishSingleFile=true and narrow it down to a smaller list?

safern · 2020-10-02T18:26:30Z

We have been doing similar things for Wasm and iOS as we enable more projects and get them green:

runtime/src/libraries/tests.proj

Line 24 in a986fe1

    
           <ItemGroup Condition="('$(TargetOS)' == 'iOS' or '$(TargetOS)' == 'tvOS') and '$(RunDisablediOSTests)' != 'true'">

Maybe we could do it in a cleaner way by adding these items to a testexclusion.props file or something and import it so that tests.proj doesn't get that complicated/polluted.

eng/pipelines/runtime.yml

safern · 2020-10-26T17:36:14Z

eng/testing/tests.singlefile.targets

+  <Target Name="__ExcludeAssembliesFromSingleFile"
+          Inputs="%(ResolvedFileToPublish.Identity)"
+          Outputs="__NewResolvedFiles"
+          BeforeTargets="_ComputeFilesToBundle">


We usually try to not depend on private targets which are implementation detail and could change their name or be removed. Is there a public target that runs before this or on the chain of this target that you can hook to?

Not at the moment, but I think we could probably add something. I'm open to adding a lot more integration points here to help with testing.

agocke · 2020-11-09T21:20:20Z

@safern @ViktorHofer Can you help me write the yaml for a new leg? I don't really know what these commands do.

safern · 2020-11-10T19:12:14Z

@safern @ViktorHofer Can you help me write the yaml for a new leg? I don't really know what these commands do.

I can help. Feel free to ping me offline.

eng/testing/tests.singlefile.targets

agocke · 2020-11-17T18:01:49Z

@safern I think I'm really close to having this working -- I think the problem is that I'm depending on the singlefilehost produced by the host partition in the libs.test build, but I think the host partition isn't necessarily being built before that partition. Is there any way to force that, or at least validate that my hypothesis is correct?

safern · 2020-11-17T19:06:08Z

@safern I think I'm really close to having this working -- I think the problem is that I'm depending on the singlefilehost produced by the host partition in the libs.test build, but I think the host partition isn't necessarily being built before that partition. Is there any way to force that, or at least validate that my hypothesis is correct?

I believe that wouldn't be such a hard change to do. @jkoritzinsky of @ViktorHofer do you know how we can change the order so that libs.tests is the last subset built when passed as part of the subsets?

safern · 2020-11-17T19:08:17Z

I believe it is just a matter of moving this:

runtime/eng/Subsets.props

Lines 245 to 247 in e5c7168

    
           <ItemGroup Condition="$(_subset.Contains('+libs.tests+'))"> 
        
             <ProjectToBuild Include="$(LibrariesProjectRoot)tests.proj" Category="libs" Test="true" /> 
        
           </ItemGroup>

to after the host subsets.

@agocke do you need all the host subsets? I'm asking as it seems like it builds host.tests by default:

https://github.com/dotnet/runtime/blob/master/eng/Subsets.props#L109-L113

agocke · 2020-11-17T19:54:47Z

I probably need host.native only

eng/Subsets.props

agocke · 2020-11-19T18:01:12Z

@safern @ViktorHofer I believe this is ready for review

The VS test work is helpful, but for now we'll have to always have a custom runner for platforms like single-file and mobile

agocke · 2021-03-29T22:31:15Z

@MichalStrehovsky I believe this is ready for integration if you wouldn't mind reviewing it.

MichalStrehovsky

Thank you!

eng/testing/runtimeConfiguration.targets

src/native/corehost/hostmisc/utils.h

eng/testing/tests.singlefile.targets

src/libraries/tests.proj

src/libraries/Common/tests/SingleFileTestRunner/SingleFileTestRunner.cs

…Runner.cs Co-authored-by: Michal Strehovský <MichalStrehovsky@users.noreply.github.com>

Co-authored-by: Michal Strehovský <MichalStrehovsky@users.noreply.github.com>

…ile-xunit

Co-authored-by: Michal Strehovský <MichalStrehovsky@users.noreply.github.com>

…ile-xunit

MichalStrehovsky

Thank you! Not a pipeline expert, so I'm not reviewing the yml changes.

ghost · 2021-03-31T17:04:18Z

Hello @agocke!

Because this pull request has the auto-merge label, I will be glad to assist with helping to merge this pull request once all check-in policies pass.

p.s. you can customize the way I help with merging this pull request, such as holding this pull request until a specific person approves. Simply @mention me (`@msftbot`) and give me an instruction to get started! Learn more here.

am11 · 2021-03-31T18:33:05Z

eng/pipelines/runtime.yml

+    buildConfig: Release
+    platforms:
+    - windows_x64
+    - Linux_x64


It seems to be missing osx_x64, is it by design?

There were some problems what I tested this earlier. Bringing Mac online would be good, but wasn't necessary for a first pass here.

Dotnet-GitSync-Bot added the area-Infrastructure-coreclr label Oct 2, 2020

agocke requested review from ViktorHofer, safern and MichalStrehovsky October 2, 2020 07:03

agocke force-pushed the single-file-xunit branch from 32bf748 to e4ce7b2 Compare October 2, 2020 07:10

SaavyGirl approved these changes Oct 2, 2020

View reviewed changes

ViktorHofer previously requested changes Oct 2, 2020

View reviewed changes

safern reviewed Oct 2, 2020

View reviewed changes

src/libraries/Common/tests/SingleFileTestRunner/SingleFileTestRunner.cs Outdated Show resolved Hide resolved

safern reviewed Oct 2, 2020

View reviewed changes

src/libraries/System.Reflection/tests/System.Reflection.Tests.csproj Outdated Show resolved Hide resolved

jkotas mentioned this pull request Oct 5, 2020

Fix SingleFile regression in IsolatedStorageFile #42306

Closed

agocke commented Oct 26, 2020

View reviewed changes

eng/pipelines/runtime.yml Show resolved Hide resolved

safern reviewed Oct 26, 2020

View reviewed changes

This was referenced Nov 12, 2020

Testing out the new system jaredpar/runfo#71

Open

Bundle_extraction_is_reused intermittently fails with "text file is busy" #44657

Closed

NuGet restore failure reading feed information #43696

Closed

safern reviewed Nov 16, 2020

View reviewed changes

eng/testing/tests.singlefile.targets Outdated Show resolved Hide resolved

safern reviewed Nov 17, 2020

View reviewed changes

eng/Subsets.props Outdated Show resolved Hide resolved

safern reviewed Nov 18, 2020

View reviewed changes

eng/Subsets.props Outdated Show resolved Hide resolved

runfoapp bot mentioned this pull request Nov 19, 2020

Bundle_Is_Extracted test failing intermittently on Linux #43316

Closed

agocke and others added 6 commits March 26, 2021 17:04

Fix subsets

a367ee0

Typo

a47ea12

Fix

5490021

Adjust assert

924892a

Include code from Michal to skip failing test

b42129c

Remove empty ItemGroup

8b733b2

MichalStrehovsky reviewed Mar 30, 2021

View reviewed changes

agocke and others added 11 commits March 30, 2021 10:32

Update src/libraries/Common/tests/SingleFileTestRunner/SingleFileTest…

1ceb9a1

…Runner.cs Co-authored-by: Michal Strehovský <MichalStrehovsky@users.noreply.github.com>

Update eng/testing/tests.singlefile.targets

6486a3a

Co-authored-by: Michal Strehovský <MichalStrehovsky@users.noreply.github.com>

Apply suggestions from code review

9b36582

Co-authored-by: Michal Strehovský <MichalStrehovsky@users.noreply.github.com>

Use ProjectExclusions

dc9a90f

Merge remote-tracking branch 'upstream/main' into single-file-xunit

d47b11e

Merge remote-tracking branch 'origin/single-file-xunit' into single-f…

a4d37ec

…ile-xunit

Update eng/testing/tests.singlefile.targets

a3d6b3b

Co-authored-by: Michal Strehovský <MichalStrehovsky@users.noreply.github.com>

Revert changes

315958b

Remove host build from tests

202daa6

Merge remote-tracking branch 'origin/single-file-xunit' into single-f…

2ad8175

…ile-xunit

Merge remote-tracking branch 'upstream/main' into single-file-xunit

d719b53

MichalStrehovsky approved these changes Mar 30, 2021

View reviewed changes

agocke added the auto-merge label Mar 31, 2021

am11 reviewed Mar 31, 2021

View reviewed changes

ghost merged commit 7677f7d into dotnet:main Mar 31, 2021

agocke deleted the single-file-xunit branch March 31, 2021 22:44

ghost locked as resolved and limited conversation to collaborators Apr 30, 2021

karelz added this to the 6.0.0 milestone May 20, 2021

This pull request was closed.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add option for building a test exe as single file #42972

Add option for building a test exe as single file #42972

agocke commented Oct 2, 2020

ViktorHofer left a comment

marek-safar commented Oct 2, 2020

agocke commented Oct 2, 2020

safern commented Oct 2, 2020

safern Oct 26, 2020

agocke Oct 27, 2020

agocke commented Nov 9, 2020

safern commented Nov 10, 2020

agocke commented Nov 17, 2020

safern commented Nov 17, 2020

safern commented Nov 17, 2020

agocke commented Nov 17, 2020

agocke commented Nov 19, 2020

agocke commented Mar 29, 2021

MichalStrehovsky left a comment

MichalStrehovsky left a comment

ghost commented Mar 31, 2021

am11 Mar 31, 2021

agocke Mar 31, 2021

Add option for building a test exe as single file #42972

Add option for building a test exe as single file #42972

Conversation

agocke commented Oct 2, 2020

ViktorHofer left a comment

Choose a reason for hiding this comment

marek-safar commented Oct 2, 2020

agocke commented Oct 2, 2020

safern commented Oct 2, 2020

safern Oct 26, 2020

Choose a reason for hiding this comment

agocke Oct 27, 2020

Choose a reason for hiding this comment

agocke commented Nov 9, 2020

safern commented Nov 10, 2020

agocke commented Nov 17, 2020

safern commented Nov 17, 2020

safern commented Nov 17, 2020

agocke commented Nov 17, 2020

agocke commented Nov 19, 2020

agocke commented Mar 29, 2021

MichalStrehovsky left a comment

Choose a reason for hiding this comment

MichalStrehovsky left a comment

Choose a reason for hiding this comment

ghost commented Mar 31, 2021

p.s. you can customize the way I help with merging this pull request, such as holding this pull request until a specific person approves. Simply @mention me (@msftbot) and give me an instruction to get started! Learn more here.

am11 Mar 31, 2021

Choose a reason for hiding this comment

agocke Mar 31, 2021

Choose a reason for hiding this comment

p.s. you can customize the way I help with merging this pull request, such as holding this pull request until a specific person approves. Simply @mention me (`@msftbot`) and give me an instruction to get started! Learn more here.