[browser][mt] Unify multithreading switches

[ilonatommy]

We used to have a runtime build switch: /p:MonoWasmBuildVariant=multithread and a public property WasmEnableThreads. When running library tests it was required to pass both switches, otherwise we got an error:

fail: [out of order message from the browser]: http://127.0.0.1:65215/_framework/dotnet.js 1759:18 Uncaught Error: Assert failed: SharedArrayBuffer is not enabled on.ers. See also https://aka.ms/dotnet-wasm-features

This PR removes MonoWasmBuildVariant and replaces it with WasmEnableThreads that will be used for runtime build and running mt-based apps / lib tests.
Rename FEATURE_WASM_THREADS -> FEATURE_WASM_MANAGED_THREADS, FeatureWasmThreads -> FeatureWasmManagedThreads. Removing MonoWasmThreads.

[ghost]

Tagging subscribers to 'arch-wasm': @lewing
See info in area-owners.md if you want to be subscribed.

Issue Details

---

We used to have a runtime build switch: /p:MonoWasmBuildVariant=multithread and a public property WasmEnableThreads. When running library tests it was required to pass both switches, otherwise we got an error:

fail: [out of order message from the browser]: http://127.0.0.1:65215/_framework/dotnet.js 1759:18 Uncaught Error: Assert failed: SharedArrayBuffer is not enabled on.ers. See also https://aka.ms/dotnet-wasm-features

This PR removes MonoWasmBuildVariant and replaces it with WasmEnableThreads that will be used for runtime build and running mt-based apps / lib tests.

Author:	ilonatommy
Assignees:	ilonatommy
Labels:	arch-wasm, area-VM-threading-mono
Milestone:	-

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[maraf]

Looks good to me 👍

[pavelsavara]

/azp run runtime-wasm

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[ilonatommy]

/azp run runtime-wasm

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[radical]

This has likely broken wasm perf builds - https://dev.azure.com/dnceng/7ea9116e-9fac-403d-b258-b31fcf1bb293/_apis/build/builds/2370299/logs/282 .
cc @LoopedBard3

[kg]

It seems like this also broke WBT for me locally:

  ** Preparing /home/kate/Projects/dotnet-runtime-wasm/artifacts/bin/dotnet-latest **
      - wasm-tools: Installing workload
      - wasm-experimental: Installing workload
/home/kate/Projects/dotnet-runtime-wasm/eng/testing/workloads-testing.targets(222,5): warning : Workload installation failed: Version 9.0.0-dev of package microsoft.netcore.app.runtime.mono.multithread.browser-wasm is not found in NuGet feeds /home/kate/Projects/dotnet-runtime-wasm/artifacts/packages/Release/Shipping/;https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-public/nuget/v3/index.json;https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json;https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json;https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-libraries/nuget/v3/index.json;https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-libraries-transport/nuget/v3/index.json;https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet9/nuget/v3/index.json;https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet9-transport/nuget/v3/index.json". [/home/kate/Projects/dotnet-runtime-wasm/src/mono/wasm/Wasm.Build.Tests/Wasm.Build.Tests.csproj]
/home/kate/Projects/dotnet-runtime-wasm/eng/testing/workloads-testing.targets(222,5): error : workload install failed with exit code 1: Installing pack Microsoft.NET.Runtime.WebAssembly.Templates version 9.0.0-dev... [/home/kate/Projects/dotnet-runtime-wasm/src/mono/wasm/Wasm.Build.Tests/Wasm.Build.Tests.csproj]
/home/kate/Projects/dotnet-runtime-wasm/eng/testing/workloads-testing.targets(222,5): error : Skipping NuGet package signature verification. [/home/kate/Projects/dotnet-runtime-wasm/src/mono/wasm/Wasm.Build.Tests/Wasm.Build.Tests.csproj]
/home/kate/Projects/dotnet-runtime-wasm/eng/testing/workloads-testing.targets(222,5): error : Writing workload pack installation record for Microsoft.NET.Runtime.WebAssembly.Templates version 9.0.0-dev... [/home/kate/Projects/dotnet-runtime-wasm/src/mono/wasm/Wasm.Build.Tests/Wasm.Build.Tests.csproj]
/home/kate/Projects/dotnet-runtime-wasm/eng/testing/workloads-testing.targets(222,5): error : Installing pack Microsoft.NETCore.App.Runtime.Mono.multithread.browser-wasm version 9.0.0-dev... [/home/kate/Projects/dotnet-runtime-wasm/src/mono/wasm/Wasm.Build.Tests/Wasm.Build.Tests.csproj]
/home/kate/Projects/dotnet-runtime-wasm/eng/testing/workloads-testing.targets(222,5): error : Workload installation failed. Rolling back installed packs... [/home/kate/Projects/dotnet-runtime-wasm/src/mono/wasm/Wasm.Build.Tests/Wasm.Build.Tests.csproj]
/home/kate/Projects/dotnet-runtime-wasm/eng/testing/workloads-testing.targets(222,5): error : Rolling back pack Microsoft.NETCore.App.Runtime.Mono.multithread.browser-wasm installation... [/home/kate/Projects/dotnet-runtime-wasm/src/mono/wasm/Wasm.Build.Tests/Wasm.Build.Tests.csproj]
/home/kate/Projects/dotnet-runtime-wasm/eng/testing/workloads-testing.targets(222,5): error : Rolling back pack Microsoft.NET.Runtime.WebAssembly.Templates installation... [/home/kate/Projects/dotnet-runtime-wasm/src/mono/wasm/Wasm.Build.Tests/Wasm.Build.Tests.csproj]
/home/kate/Projects/dotnet-runtime-wasm/eng/testing/workloads-testing.targets(222,5): error : Uninstalling workload pack Microsoft.NET.Runtime.WebAssembly.Templates version 9.0.0-dev... [/home/kate/Projects/dotnet-runtime-wasm/src/mono/wasm/Wasm.Build.Tests/Wasm.Build.Tests.csproj]
/home/kate/Projects/dotnet-runtime-wasm/eng/testing/workloads-testing.targets(222,5): error : Workload installation failed: Version 9.0.0-dev of package microsoft.netcore.app.runtime.mono.multithread.browser-wasm is not found in NuGet feeds /home/kate/Projects/dotnet-runtime-wasm/artifacts/packages/Release/Shipping/;https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-public/nuget/v3/index.json;https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json;https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json;https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-libraries/nuget/v3/index.json;https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-libraries-transport/nuget/v3/index.json;https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet9/nuget/v3/index.json;https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet9-transport/nuget/v3/index.json". [/home/kate/Projects/dotnet-runtime-wasm/src/mono/wasm/Wasm.Build.Tests/Wasm.Build.Tests.csproj]