Add initial interop testing framework #1932

hughbe · 2019-09-18T12:34:37Z

Proposed Changes

Add C++ interop tests for WebBrowserSiteBase

@JeremyKuhne @RussKie @zsd4yr

Microsoft Reviewers: Open in CodeFlow

src/System.Windows.Forms/tests/InteropTests/TestDll/WebBrowserSiteBaseInteropTests.cpp

src/System.Windows.Forms/tests/InteropTests/InteropTestBase.cs

codecov · 2019-10-29T12:15:06Z

Codecov Report

Merging #1932 into master will decrease coverage by 38.34061%.
The diff coverage is n/a.

@@                 Coverage Diff                  @@
##              master       #1932          +/-   ##
====================================================
- Coverage   65.14931%   26.80870%   -38.34061%     
====================================================
  Files           1328         855         -473     
  Lines         489778      266794      -222984     
  Branches       40042       37916        -2126     
====================================================
- Hits          319087       71524      -247563     
- Misses        165265      190162       +24897     
+ Partials        5426        5108         -318

Flag	Coverage Δ
#Debug	`26.80870% <ø> (-38.34061%)`	⬇️
#production	`26.80870% <ø> (-7.37191%)`	⬇️
#test	`100.00000% <ø> (+1.53383%)`	⬆️

src/System.Windows.Forms/tests/UnitTests/InteropTests/testhelpers.h

src/System.Windows.Forms/tests/UnitTests/InteropTests/WebBrowserSiteBaseInteropTests.cpp

hughbe · 2020-01-21T12:19:17Z

Waiting on dotnet/arcade#4533. Does anyone know how I can pull in a PR build of aracde so I can test getting this working with the new SDK? Do I have to have a local build, maybe

src/System.Windows.Forms/tests/Directory.Build.targets

src/System.Windows.Forms/tests/UnitTests/InteropTests/testhelpers.h

src/System.Windows.Forms/tests/Directory.Build.targets

RussKie · 2020-02-19T05:21:04Z

Just to confirm, is this work addressing #1817?

hughbe · 2020-04-19T12:13:17Z

Just to confirm, is this work addressing #1817?

It makes a start by adding a limited set of tests. Mostly the PR is designed to setup the infrastructure for adding more tests in the future

hughbe · 2020-04-28T17:47:25Z

YAY! dotnet/arcade has merged the change we need. Now we need the latest version of arcade (#3134) and we can get this PR out of WIP!!

hughbe · 2020-04-29T08:35:23Z

Looks like the latest build of arcade still isn't out :)

hughbe · 2020-05-01T10:27:46Z

Okee dokee, we have progress! I've rebased the PR onto the latest master, which has Microsoft.DotNet.CMake.SDK

However we now have the following error in build:

.packages\microsoft.dotnet.cmake.sdk\5.0.0-beta.20228.4\build\Microsoft.DotNet.CMake.Sdk.targets(29,5): error : (NETCORE_ENGINEERING_TELEMETRY=Build) A CMake project must be built for a specific architecture.

I suspect it is because we're building the c# test project as AnyCPU but should be building it for x86 and x64 separately.

Also I would add it wasn't clear that I had to include Microsoft.DotNet.CMake.Sdk in global.json :))

hughbe · 2020-06-16T07:29:16Z

Yeah sadly adding to the path doesn’t seem to work anymore now that we init the tools only once. It may be possible that the path changed. Will Check after my exam today

hughbe · 2020-06-16T08:49:24Z

Lets see if a similar change to 32eaa68 changing from tools/native/bin -> tools/bin will work

hughbe · 2020-07-27T15:13:37Z

@RussKie could you take another look at this?

RussKie · 2020-07-28T02:45:06Z

@vatsan-madhavan any further concerns?

vatsan-madhavan · 2020-07-28T04:46:28Z

@vatsan-madhavan any further concerns?

lgtm.

RussKie · 2020-07-28T08:29:24Z

could you take another look at this?

I'm probably with the least of knowledge in this area here. Everyone seem to give thumbs up, and the build is green, so
We can always iterate further and tweak, if necessary.

A massive feat, thank you 🚀

vatsan-madhavan · 2020-07-28T19:26:22Z

Nice job @hughbe!

hughbe · 2020-07-29T08:17:00Z

Igor what is your version of cmake (cmake --version)? This looks similar to dotnet/runtime#660 so perhaps updating your cmake might work

vladimir-krestov · 2020-07-29T09:49:35Z

@hughbe, I had cmake version 3.17.20032601-MSVC_2 as now too. Before I had a different error in build.cmd:

But I just updated VisualStudio ✔️ , restart PC and now build works well:

FYI: @RussKie

RussKie · 2020-07-29T10:11:26Z

C:\Development\winforms>cmake --version
cmake version 3.17.20032601-MSVC_2

I'm also on the internal preview of VS Version 16.8.0 Preview 2.0 [30324.147.master].... 🤔

RussKie · 2020-07-29T10:59:32Z

I have updated VS and rebooted... No joy.
Can't build from VS either:

33>------ Rebuild All started: Project: System.Windows.Forms.Interop.Tests, Configuration: Debug x86 ------
32>C:\Development\winforms\src\System.Windows.Forms.Design\tests\UnitTests\System\ComponentModel\Design\Serialization\CodeDomSerializerExceptionTests.cs(99,61,99,99): warning SYSLIB0011: 'BinaryFormatter.Serialize(Stream, object)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'
32>C:\Development\winforms\src\System.Windows.Forms.Design\tests\UnitTests\System\ComponentModel\Design\Serialization\CodeDomComponentSerializationServiceTests.cs(808,61,808,95): warning SYSLIB0011: 'BinaryFormatter.Serialize(Stream, object)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'
32>C:\Development\winforms\src\System.Windows.Forms.Design\tests\UnitTests\System\ComponentModel\Design\ExceptionCollectionTests.cs(49,61,49,100): warning SYSLIB0011: 'BinaryFormatter.Serialize(Stream, object)' is obsolete: 'BinaryFormatter serialization is obsolete and should not be used. See https://aka.ms/binaryformatter for more information.'
32>System.Windows.Forms.Design.Tests -> C:\Development\winforms\artifacts\bin\System.Windows.Forms.Design.Tests\Debug\netcoreapp5.0\System.Windows.Forms.Design.Tests.dll
32>Done building project "System.Windows.Forms.Design.Tests.csproj".
33>System.Windows.Forms.Interop.Tests -> C:\Development\winforms\artifacts\bin\System.Windows.Forms.Interop.Tests\x86\Debug\netcoreapp5.0\System.Windows.Forms.Interop.Tests.dll
33>'cmake' is not recognized as an internal or external command,
33>operable program or batch file.
33>C:\Users\igveliko\.nuget\packages\microsoft.dotnet.cmake.sdk\5.0.0-beta.20256.5\build\Microsoft.DotNet.CMake.Sdk.targets(172,5): error MSB3073: The command "
33>C:\Users\igveliko\.nuget\packages\microsoft.dotnet.cmake.sdk\5.0.0-beta.20256.5\build\Microsoft.DotNet.CMake.Sdk.targets(172,5): error MSB3073:       
33>C:\Users\igveliko\.nuget\packages\microsoft.dotnet.cmake.sdk\5.0.0-beta.20256.5\build\Microsoft.DotNet.CMake.Sdk.targets(172,5): error MSB3073:       
33>C:\Users\igveliko\.nuget\packages\microsoft.dotnet.cmake.sdk\5.0.0-beta.20256.5\build\Microsoft.DotNet.CMake.Sdk.targets(172,5): error MSB3073:        cmake -G "Visual Studio 16 2019" -A Win32 -B "C:\Development\winforms\artifacts\obj\NativeTests\x86\Debug" -S "" -DCMAKE_INSTALL_PREFIX=C:\Development\winforms\artifacts\bin\NativeTests\x86\Debug
33>C:\Users\igveliko\.nuget\packages\microsoft.dotnet.cmake.sdk\5.0.0-beta.20256.5\build\Microsoft.DotNet.CMake.Sdk.targets(172,5): error MSB3073:       " exited with code 9009.
33>Done building project "NativeTests.proj" -- FAILED.
Build has been canceled.

vladimir-krestov · 2020-07-29T11:12:35Z

Sometimes I see this problem in VS too.

RussKie · 2020-07-29T11:23:03Z

I think I had a headway.

I found cmake.exe and a cmake folder in winforms\.tools\native\bin folder. cmake.exe was dated 10 Jul 2020, and the folder had cmake 3.14.2...
I have deleted both, and it appears to work now.

I have also nuked the whole .tools folder, and it seemed to make no difference to the build ¯\_(ツ)_/¯

RussKie · 2020-07-29T11:37:02Z

VS can't find cmake, so we'll need to tweak start-vs.cmd to resolve and add the path to cmake, I guess the Arcade build scripts do it internally.

In mean time I solved it by installing cmake globally: choco install cmake --installargs 'ADD_CMAKE_TO_PATH=System' and reboot.

vatsan-madhavan · 2020-07-29T16:46:40Z

See https://aka.ms/vs/workloads - Microsoft.VisualStudio.Component.VC.CMake.Project

Can you add that to your VS installation and then try again ? Once you've figured out a min-set of workloads needed to build dotnet/winforms, document it like this.

RussKie · 2020-07-30T02:55:19Z

Thank you @vatsan-madhavan, I knew it was possible just didn't know how.

hughbe requested a review from a team as a code owner September 18, 2019 12:34

RussKie reviewed Sep 18, 2019

View reviewed changes

src/System.Windows.Forms/tests/InteropTests/TestDll/WebBrowserSiteBaseInteropTests.cpp Outdated Show resolved Hide resolved

src/System.Windows.Forms/tests/InteropTests/InteropTestBase.cs Outdated Show resolved Hide resolved

RussKie assigned hughbe Sep 24, 2019

RussKie added the test-enhancement Improvements of test source code label Sep 24, 2019

hughbe mentioned this pull request Sep 24, 2019

Need a common way to reference CMakeLists.txt projects from a csproj dotnet/arcade#4008

Closed

hughbe force-pushed the interop-tests branch from 335c827 to f8c0bd0 Compare October 29, 2019 12:14

hughbe force-pushed the interop-tests branch from f8c0bd0 to a7f69e8 Compare October 29, 2019 13:50

AaronRobinsonMSFT reviewed Oct 29, 2019

View reviewed changes

weltkante reviewed Oct 29, 2019

View reviewed changes

src/System.Windows.Forms/tests/UnitTests/InteropTests/WebBrowserSiteBaseInteropTests.cpp Outdated Show resolved Hide resolved

hughbe force-pushed the interop-tests branch from a7f69e8 to bdc8137 Compare November 2, 2019 11:23

hughbe force-pushed the interop-tests branch from bdc8137 to fb5de72 Compare January 6, 2020 22:01

hughbe force-pushed the interop-tests branch from fb5de72 to 2a9e2df Compare January 21, 2020 12:17

hughbe force-pushed the interop-tests branch from 2a9e2df to 87ad984 Compare January 23, 2020 18:36

RussKie reviewed Jan 31, 2020

View reviewed changes

RussKie mentioned this pull request Feb 19, 2020

Cleanup EDITSTREAM #2816

Merged

hughbe force-pushed the interop-tests branch from 87ad984 to 58ee0f0 Compare March 12, 2020 17:31

hughbe mentioned this pull request Apr 19, 2020

Add Microsoft.DotNet.CMake.Sdk MSBuild SDK for integrating native builds into MSBuild. dotnet/arcade#4533

Merged

hughbe force-pushed the interop-tests branch from 58ee0f0 to 337b4e0 Compare April 28, 2020 17:46

hughbe force-pushed the interop-tests branch from 337b4e0 to dfae6a6 Compare April 29, 2020 07:46

hughbe force-pushed the interop-tests branch from dfae6a6 to de152ab Compare May 1, 2020 09:21

hughbe changed the title ~~[WIP] Add initial interop testing framework~~ Add initial interop testing framework May 1, 2020

hughbe force-pushed the interop-tests branch from de152ab to 2986fef Compare May 1, 2020 09:28

hughbe force-pushed the interop-tests branch from 2986fef to f7569cc Compare May 1, 2020 11:55

hughbe mentioned this pull request Jun 16, 2020

ErrorProviderAccessibleObjectTests.Ctor tests flakey #3391

Closed

hughbe force-pushed the interop-tests branch from 7b4fe01 to d4f066c Compare June 16, 2020 09:08

This comment has been minimized.

Sign in to view

hughbe force-pushed the interop-tests branch from d4f066c to 11e99c3 Compare June 16, 2020 11:17

weltkante mentioned this pull request Jun 16, 2020

Flaky Clipboard Unit Tests #3456

Closed

hughbe force-pushed the interop-tests branch 2 times, most recently from 9cacc7a to d64a50a Compare July 8, 2020 13:43

Add initial interop testing framework

ef049af

hughbe force-pushed the interop-tests branch from d64a50a to ef049af Compare July 27, 2020 15:18

RussKie merged commit 3bae4e6 into dotnet:master Jul 28, 2020

ghost added this to the 5.0 RC1 milestone Jul 28, 2020

hughbe deleted the interop-tests branch July 28, 2020 08:32

This comment has been minimized.

Sign in to view

ghost locked as resolved and limited conversation to collaborators Feb 4, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add initial interop testing framework #1932

Add initial interop testing framework #1932

hughbe commented Sep 18, 2019 •

edited

Loading

codecov bot commented Oct 29, 2019 •

edited

Loading

hughbe commented Jan 21, 2020

RussKie commented Feb 19, 2020

hughbe commented Apr 19, 2020

hughbe commented Apr 28, 2020

hughbe commented Apr 29, 2020

hughbe commented May 1, 2020 •

edited

Loading

hughbe commented Jun 16, 2020

hughbe commented Jun 16, 2020

This comment has been minimized.

hughbe commented Jul 27, 2020

RussKie commented Jul 28, 2020

vatsan-madhavan commented Jul 28, 2020

RussKie commented Jul 28, 2020

vatsan-madhavan commented Jul 28, 2020

This comment has been minimized.

hughbe commented Jul 29, 2020

vladimir-krestov commented Jul 29, 2020

RussKie commented Jul 29, 2020

RussKie commented Jul 29, 2020

vladimir-krestov commented Jul 29, 2020

RussKie commented Jul 29, 2020 •

edited

Loading

RussKie commented Jul 29, 2020

vatsan-madhavan commented Jul 29, 2020

RussKie commented Jul 30, 2020

Add initial interop testing framework #1932

Add initial interop testing framework #1932

Conversation

hughbe commented Sep 18, 2019 • edited Loading

Proposed Changes

Microsoft Reviewers: Open in CodeFlow

codecov bot commented Oct 29, 2019 • edited Loading

Codecov Report

hughbe commented Jan 21, 2020

RussKie commented Feb 19, 2020

hughbe commented Apr 19, 2020

hughbe commented Apr 28, 2020

hughbe commented Apr 29, 2020

hughbe commented May 1, 2020 • edited Loading

hughbe commented Jun 16, 2020

hughbe commented Jun 16, 2020

This comment has been minimized.

hughbe commented Jul 27, 2020

RussKie commented Jul 28, 2020

vatsan-madhavan commented Jul 28, 2020

RussKie commented Jul 28, 2020

vatsan-madhavan commented Jul 28, 2020

This comment has been minimized.

hughbe commented Jul 29, 2020

vladimir-krestov commented Jul 29, 2020

RussKie commented Jul 29, 2020

RussKie commented Jul 29, 2020

vladimir-krestov commented Jul 29, 2020

RussKie commented Jul 29, 2020 • edited Loading

RussKie commented Jul 29, 2020

vatsan-madhavan commented Jul 29, 2020

RussKie commented Jul 30, 2020

hughbe commented Sep 18, 2019 •

edited

Loading

codecov bot commented Oct 29, 2019 •

edited

Loading

hughbe commented May 1, 2020 •

edited

Loading

RussKie commented Jul 29, 2020 •

edited

Loading