[wasm] Add support for using custom native libraries #55797

radical · 2021-07-16T04:50:37Z

Native libraries can be referenced via @(NativeReference) item, eg:

<NativeFileReference Include="NativeLib.o" />

JS files can be added via @(Content), with a Kind metadata, eg:

<Content Include="foo.js" Kind="pre-js" />

Native libraries can be referenced via `@(NativeReference)` item, eg: `<NativeReference Include="NativeLib.o" />` - JS files can be added via `@(Content)`, with a `Kind` metadata, eg: `<Content Include="foo.js" Kind="pre-js" />`

ghost · 2021-07-16T04:50:43Z

Tagging subscribers to 'arch-wasm': @lewing
See info in area-owners.md if you want to be subscribed.

Issue Details

Native libraries can be referenced via @(NativeReference) item, eg:

<NativeReference Include="NativeLib.o" />

JS files can be added via @(Content), with a Kind metadata, eg:

<Content Include="foo.js" Kind="pre-js" />

Author:	radical
Assignees:	-
Labels:	`arch-wasm`, `area-Build-mono`
Milestone:	-

lewing · 2021-07-16T13:47:48Z

failure is


Error message
System.ComponentModel.Win32Exception : Text file busy


Stack trace
   at System.Diagnostics.Process.ForkAndExecProcess(String filename, String[] argv, String[] envp, String cwd, Boolean redirectStdin, Boolean redirectStdout, Boolean redirectStderr, Boolean setCredentials, UInt32 userId, UInt32 groupId, UInt32[] groups, Int32& stdinFd, Int32& stdoutFd, Int32& stderrFd, Boolean usesTerminal, Boolean throwOnNoExec) in System.Diagnostics.Process.dll:token 0x6000122+0xad
   at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo) in System.Diagnostics.Process.dll:token 0x6000121+0x1aa
   at System.Diagnostics.Process.Start() in System.Diagnostics.Process.dll:token 0x60000fa+0xab
   at Microsoft.DotNet.Cli.Build.Framework.Command.Start() in /_/src/installer/tests/TestUtils/Command.cs:line 199
   at Microsoft.DotNet.Cli.Build.Framework.Command.Execute(Boolean fExpectedToFail) in /_/src/installer/tests/TestUtils/Command.cs:line 239
   at Microsoft.DotNet.Cli.Build.Framework.Command.Execute() in /_/src/installer/tests/TestUtils/Command.cs:line 171
   at AppHost.Bundle.Tests.NetCoreApp3CompatModeTests.Bundle_Is_Extracted() in /_/src/installer/tests/Microsoft.NET.HostModel.Tests/AppHost.Bundle.Tests/NetCoreApp3CompatModeTests.cs:line 33

https://dev.azure.com/dnceng/public/_build/results?buildId=1241841&view=ms.vss-test-web.build-test-results-tab&runId=36929180&resultId=100004&paneView=debug

.. and update the tests to use that.

lewing · 2021-07-19T01:19:01Z

/azp run runtime,runtime-staging

azure-pipelines · 2021-07-19T01:19:38Z

Azure Pipelines successfully started running 2 pipeline(s).

(cherry picked from commit d574b03)

…raries (#56013) * [wasm] Add support for using custom native libraries (#55797) (cherry picked from commit d574b03) * [wasm] Use compile rsp instead of link, for compiling native files (#55848) .. and fix logging that broke recently. `tasks/Common/Utils.cs`: TaskLoggingHelper Utils.Logger is a static field, which must be set by task else any methods in Utils, eg. RunProcess, silently fail to log any messages. Also, this would be a problem when building multiple projects in parallel, since the logger is a task-specific one. Instead, we pass logger as an arg to all the methods. (cherry picked from commit 3301e9d) * Link with EmccCompileOptimizationFlag==-Oz by default in release (#55939) (cherry picked from commit 04072ff) * [wasm] Fix regression in compiling .bc -> .o files (#56063) * [wasm] Add back --emit-llvm that got removed mistakenly, in an earlier commit .. found thanks to Jerome Laban. * [wasm] Set EmccCompile's messages to MessageImportance.Low by default. .. and to MessageImportance.Normal if `$(EmccVerbose)==true`. * [wasm] Quote filenames passed to emcc compile command line * Add more blazorwasm tests - for debug/release, aot/relinking * Bump sdk for workload testing to 6.0.100-rc.1.21370.2 * [wasm] Fix regression in compiling bitcode -> .o The `-emit-llvm` arg has been incorrectly added, and removed from the args used for compiling .bc->.o . This commit fixes it, and adds a crude test for it, so we don't regress again. * Fix build (cherry picked from commit 1d8ad03) * [wasm] Bump sdk for workload testing to 6.0.100-preview.7.21372.19 Co-authored-by: Larry Ewing <lewing@microsoft.com>

TL;dr `publish` fails on any blazorwasm project with VS17 A recent commit[1] moved initializing `$(_WasmIntermediateOutputPath)` from a target, to project level `PropertyGroup`. It is set as: `<_WasmIntermediateOutputPath>$([MSBuild]::NormalizeDirectory($(IntermediateOutputPath), 'wasm'))</_WasmIntermediateOutputPath>` The `NormalizeDirectory` call converts this to a full path, presumably using the current directory. Because we are setting `$(_WasmIntermediateOutputPath)` at the project level, it gets evaluated during the evaluation phase. And the current directory doesn't seem to be set to the project directory at that point in VS. So, `$(_WasmIntermediateOutputPath)` gets a wrong path like: `C:\Program Files\Microsoft Visual Studio\2022\Preview\Common7\IDE\obj\Release\net6.0\wasm`. And then when actually publishing, it fails to create this directory with: `Unable to create directory "C:\Program Files\Microsoft Visual Studio\2022\Preview\Common7\IDE\obj\Release\net6.0\wasm\". Access to the path 'C:\Program Files\Microsoft Visual Studio\2022\Preview\Common7\IDE\obj\Release\net6.0\wasm\' is denied.` Fix: Set the property in `_InitializeCommonProperties` *target*, at which point the current directory is correctly set. Note: - This doesn't seem to be reproducible outside VS - It happens only on `publish`, because that's when the wasm targets come into play. -- 1. ``` commit d574b03 Author: Ankit Jain <radical@gmail.com> Date: Mon Jul 19 01:02:01 2021 -0400 [wasm] Add support for using custom native libraries (#55797) ```

TL;dr `publish` fails on any blazorwasm project with VS17 A recent commit[1] moved initializing `$(_WasmIntermediateOutputPath)` from a target, to project level `PropertyGroup`. It is set as: `<_WasmIntermediateOutputPath>$([MSBuild]::NormalizeDirectory($(IntermediateOutputPath), 'wasm'))</_WasmIntermediateOutputPath>` The `NormalizeDirectory` call converts this to a full path, presumably using the current directory. Because we are setting `$(_WasmIntermediateOutputPath)` at the project level, it gets evaluated during the evaluation phase. And the current directory doesn't seem to be set to the project directory at that point in VS. So, `$(_WasmIntermediateOutputPath)` gets a wrong path like: `C:\Program Files\Microsoft Visual Studio\2022\Preview\Common7\IDE\obj\Release\net6.0\wasm`. And then when actually publishing, it fails to create this directory with: `Unable to create directory "C:\Program Files\Microsoft Visual Studio\2022\Preview\Common7\IDE\obj\Release\net6.0\wasm\". Access to the path 'C:\Program Files\Microsoft Visual Studio\2022\Preview\Common7\IDE\obj\Release\net6.0\wasm\' is denied.` Fix: Set the property in `_InitializeCommonProperties` *target*, at which point the current directory is correctly set. Note: - This doesn't seem to be reproducible outside VS - It happens only on `publish`, because that's when the wasm targets come into play. -- 1. ``` commit d574b03 Author: Ankit Jain <radical@gmail.com> Date: Mon Jul 19 01:02:01 2021 -0400 [wasm] Add support for using custom native libraries (#55797) ``` Co-authored-by: Ankit Jain <radical@gmail.com>

TL;dr `publish` fails on any blazorwasm project with VS17 A recent commit[1] moved initializing `$(_WasmIntermediateOutputPath)` from a target, to project level `PropertyGroup`. It is set as: `<_WasmIntermediateOutputPath>$([MSBuild]::NormalizeDirectory($(IntermediateOutputPath), 'wasm'))</_WasmIntermediateOutputPath>` The `NormalizeDirectory` call converts this to a full path, presumably using the current directory. Because we are setting `$(_WasmIntermediateOutputPath)` at the project level, it gets evaluated during the evaluation phase. And the current directory doesn't seem to be set to the project directory at that point in VS. So, `$(_WasmIntermediateOutputPath)` gets a wrong path like: `C:\Program Files\Microsoft Visual Studio\2022\Preview\Common7\IDE\obj\Release\net6.0\wasm`. And then when actually publishing, it fails to create this directory with: `Unable to create directory "C:\Program Files\Microsoft Visual Studio\2022\Preview\Common7\IDE\obj\Release\net6.0\wasm\". Access to the path 'C:\Program Files\Microsoft Visual Studio\2022\Preview\Common7\IDE\obj\Release\net6.0\wasm\' is denied.` Fix: Set the property in `_InitializeCommonProperties` *target*, at which point the current directory is correctly set. Note: - This doesn't seem to be reproducible outside VS - It happens only on `publish`, because that's when the wasm targets come into play. -- 1. ``` commit d574b03 Author: Ankit Jain <radical@gmail.com> Date: Mon Jul 19 01:02:01 2021 -0400 [wasm] Add support for using custom native libraries (dotnet#55797) ```

TL;dr `publish` fails on any blazorwasm project with VS17 A recent commit[1] moved initializing `$(_WasmIntermediateOutputPath)` from a target, to project level `PropertyGroup`. It is set as: `<_WasmIntermediateOutputPath>$([MSBuild]::NormalizeDirectory($(IntermediateOutputPath), 'wasm'))</_WasmIntermediateOutputPath>` The `NormalizeDirectory` call converts this to a full path, presumably using the current directory. Because we are setting `$(_WasmIntermediateOutputPath)` at the project level, it gets evaluated during the evaluation phase. And the current directory doesn't seem to be set to the project directory at that point in VS. So, `$(_WasmIntermediateOutputPath)` gets a wrong path like: `C:\Program Files\Microsoft Visual Studio\2022\Preview\Common7\IDE\obj\Release\net6.0\wasm`. And then when actually publishing, it fails to create this directory with: `Unable to create directory "C:\Program Files\Microsoft Visual Studio\2022\Preview\Common7\IDE\obj\Release\net6.0\wasm\". Access to the path 'C:\Program Files\Microsoft Visual Studio\2022\Preview\Common7\IDE\obj\Release\net6.0\wasm\' is denied.` Fix: Set the property in `_InitializeCommonProperties` *target*, at which point the current directory is correctly set. Note: - This doesn't seem to be reproducible outside VS - It happens only on `publish`, because that's when the wasm targets come into play. -- 1. ``` commit d574b03 Author: Ankit Jain <radical@gmail.com> Date: Mon Jul 19 01:02:01 2021 -0400 [wasm] Add support for using custom native libraries (#55797) ```

[wasm] Add support for using custom native libraries

d68b40b

Native libraries can be referenced via `@(NativeReference)` item, eg: `<NativeReference Include="NativeLib.o" />` - JS files can be added via `@(Content)`, with a `Kind` metadata, eg: `<Content Include="foo.js" Kind="pre-js" />`

radical added arch-wasm WebAssembly architecture area-Build-mono labels Jul 16, 2021

radical requested review from lewing and vargaz July 16, 2021 04:50

radical requested a review from marek-safar as a code owner July 16, 2021 04:50

radical added 4 commits July 16, 2021 00:56

Add license header for new source files

47a4d0b

Use @(NativeFileReference) instead of @(NativeReference)

cfcaa6f

Add missing NativeLibraryTests.cs, and enable aot builds

1adbf31

Add missing NativeLib.o

7973d6f

lewing requested review from radekdoulik and akoeplinger July 16, 2021 18:47

radical force-pushed the wasm-native-libs branch from 30f7ae6 to 1ebae57 Compare July 16, 2021 21:31

PInvokeTableGenerator: handle native libs with hyphen in their name

0ea5099

.. and update the tests to use that.

radical force-pushed the wasm-native-libs branch from 1ebae57 to 0ea5099 Compare July 16, 2021 21:53

lewing approved these changes Jul 19, 2021

View reviewed changes

radical merged commit d574b03 into dotnet:main Jul 19, 2021

radical deleted the wasm-native-libs branch July 19, 2021 05:02

ManickaP mentioned this pull request Jul 20, 2021

[QUIC] Remove AppContext switch from S.N.Quic #56027

Merged

radical added a commit to radical/runtime that referenced this pull request Jul 22, 2021

[wasm] Add support for using custom native libraries (dotnet#55797)

b0c2e73

(cherry picked from commit d574b03)

radical mentioned this pull request Jul 22, 2021

[release/6.0-preview7] [wasm] Add support for using custom native libraries #56013

Merged

karelz mentioned this pull request Jul 28, 2021

MsQuic tests hang / long running #56487

Closed

ghost locked as resolved and limited conversation to collaborators Aug 18, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[wasm] Add support for using custom native libraries #55797

[wasm] Add support for using custom native libraries #55797

radical commented Jul 16, 2021 •

edited

Loading

ghost commented Jul 16, 2021

lewing commented Jul 16, 2021

lewing commented Jul 19, 2021

azure-pipelines bot commented Jul 19, 2021

[wasm] Add support for using custom native libraries #55797

[wasm] Add support for using custom native libraries #55797

Conversation

radical commented Jul 16, 2021 • edited Loading

ghost commented Jul 16, 2021

lewing commented Jul 16, 2021

lewing commented Jul 19, 2021

azure-pipelines bot commented Jul 19, 2021

radical commented Jul 16, 2021 •

edited

Loading