Skip to content

Commit

Permalink
Merge pull request #374 from jkotas/merge-master
Browse files Browse the repository at this point in the history
Merge from dotnet/runtime
  • Loading branch information
jkotas committed Nov 24, 2020
2 parents 4d76b93 + de84734 commit 7faa1f0
Show file tree
Hide file tree
Showing 597 changed files with 6,904 additions and 5,995 deletions.
2 changes: 1 addition & 1 deletion .config/dotnet-tools.json
Original file line number Diff line number Diff line change
Expand Up @@ -27,4 +27,4 @@
]
}
}
}
}
4 changes: 1 addition & 3 deletions Directory.Build.props
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@
<WasmAppBuilderDir>$([MSBuild]::NormalizeDirectory('$(ArtifactsBinDir)', 'WasmAppBuilder', 'Debug', '$(NetCoreAppCurrent)', 'publish'))</WasmAppBuilderDir>
<WasmBuildTasksDir>$([MSBuild]::NormalizeDirectory('$(ArtifactsBinDir)', 'WasmBuildTasks', 'Debug', '$(NetCoreAppCurrent)', 'publish'))</WasmBuildTasksDir>
<MonoAOTCompilerDir>$([MSBuild]::NormalizeDirectory('$(ArtifactsBinDir)', 'MonoAOTCompiler', 'Debug', '$(NetCoreAppCurrent)'))</MonoAOTCompilerDir>

<AppleAppBuilderTasksAssemblyPath>$([MSBuild]::NormalizePath('$(AppleAppBuilderDir)', 'AppleAppBuilder.dll'))</AppleAppBuilderTasksAssemblyPath>
<AndroidAppBuilderTasksAssemblyPath>$([MSBuild]::NormalizePath('$(AndroidAppBuilderDir)', 'AndroidAppBuilder.dll'))</AndroidAppBuilderTasksAssemblyPath>
<WasmAppBuilderTasksAssemblyPath>$([MSBuild]::NormalizePath('$(WasmAppBuilderDir)', 'WasmAppBuilder.dll'))</WasmAppBuilderTasksAssemblyPath>
Expand Down Expand Up @@ -106,8 +106,6 @@
</PropertyGroup>

<PropertyGroup>
<!-- Default to portable build if not explicitly set -->
<PortableBuild Condition="'$(PortableBuild)' == ''">true</PortableBuild>
<!-- Default to discarding symbols if not explicitly set -->
<KeepNativeSymbols Condition="'$(KeepNativeSymbols)' == ''">false</KeepNativeSymbols>
<!-- Used for launchSettings.json and runtime config files. -->
Expand Down
8 changes: 4 additions & 4 deletions docs/design/coreclr/jit/lsra-detail.md
Original file line number Diff line number Diff line change
Expand Up @@ -387,7 +387,7 @@ well as supporting components) in more depth.
- `LinearScan::identifyCandidates`

- This mostly duplicates what is done in
`Compiler::lvaMarkLocalVars(). There are differences in what
`Compiler::lvaMarkLocalVars()`. There are differences in what
constitutes a register candidate vs. what is tracked, but we
should probably handle them in `Compiler::lvaMarkLocalVars()`
when it is called after `Lowering`.
Expand Down Expand Up @@ -445,7 +445,7 @@ node, which builds `RefPositions` according to the liveness model described abov

- First, we create `RefPosition`s to define any internal registers that are required.
As we create new `Interval`s for these, we add the definition `RefPosition` to an array,
`InternalDefs`. This allows us to create the corresponding uses of these internal
`internalDefs`. This allows us to create the corresponding uses of these internal
registers later.

- Then we create `RefPosition`s for each use in the instruction.
Expand Down Expand Up @@ -474,7 +474,7 @@ node, which builds `RefPositions` according to the liveness model described abov
we need to ensure that the other source isn't given the same register as the
target. For this, we annotate the use `RefPosition` with `delayRegFree`.

- Next we create the uses of the internal registers, using the `InternalDefs` array.
- Next we create the uses of the internal registers, using the `internalDefs` array.
This is cleared before the next instruction is handled.

- Next, any registers in the kill set for the instruction are killed. This is performed
Expand Down Expand Up @@ -652,7 +652,7 @@ LinearScanAllocation(List<RefPosition> refPositions)
- When `COMPlus_EnableEHWriteThru == 0`, any variable that's
live in to an exception region is always referenced on the stack.

- See [Enable EHWriteThru by default](#enable-EHWriteThru-by-default).
- See [Enable EHWriteThru by default](#enable-ehwritethru-by-default).

- Code generation (genGenerateCode)

Expand Down
108 changes: 81 additions & 27 deletions docs/workflow/testing/mono/testing.md
Original file line number Diff line number Diff line change
@@ -1,54 +1,108 @@
# Running Tests using Mono Runtime
# Running test suites using Mono

## Running Runtime Tests
We currently only support running tests against coreclr. There are additional mono runtime tests in mono/mono, but they
have not been moved over yet. Simply run the following command:
Before running tests, [build Mono](../../building/mono/README.md) using the desired configuration.

## Runtime Tests
### Desktop Mono:

To build the runtime tests for Mono JIT or interpreter, execute the following command from `$(REPO_ROOT)/src/tests`
```
./build.sh excludemonofailures <release|debug>
```

Run individual test:
```
cd ../mono/netcore
make run-tests-coreclr CoreClrTest="bash ../../artifacts/tests/coreclr/OSX.x64.Release/JIT/opt/InstructionCombining/DivToMul/DivToMul.sh"
```

Run all tests:
```
cd ../mono/netcore
make run-tests-coreclr-all
```

### WebAssembly:
Build the runtime tests for WebAssembly
```
$(REPO_ROOT)/src/tests/build.sh -skipstressdependencies -excludemonofailures os Browser wasm <Release/Debug>
```

The last few lines of the build log should contain something like this:
```
dotnet build /t:RunCoreClrTests $(REPO_ROOT)/src/mono/mono.proj
--------------------------------------------------
Example run.sh command
src/tests/run.sh --coreOverlayDir=<repo_root>artifacts/tests/coreclr/Browser.wasm.Release/Tests/Core_Root --testNativeBinDir=<repo_root>/artifacts/obj/coreclr/Browser.wasm.Release/tests --testRootDir=<repo_root>/artifacts/tests/coreclr/Browser.wasm.Release --copyNativeTestBin Release
--------------------------------------------------
```

If you want to run individual tests, execute this command:
To run all tests, execute that command, adding `wasm` to the end.

### Android:
Build the runtime tests for Android x64
```
$(REPO_ROOT)/src/tests/build.sh -skipstressdependencies -excludemonofailures os Android x64 <Release/Debug>
```
dotnet build /t:RunCoreClrTest /p:CoreClrTest="<TestName>" $(REPO_ROOT)/src/mono/mono.proj

The last few lines of the build log should contain something like this:
```
--------------------------------------------------
Example run.sh command
## Running Library Tests
Running library tests against Mono is straightforward regardless of configuration. Simply run the following commands:
src/tests/run.sh --coreOverlayDir=<repo_root>artifacts/tests/coreclr/Android.x64.Release/Tests/Core_Root --testNativeBinDir=<repo_root>/artifacts/obj/coreclr/Android.x64.Release/tests --testRootDir=<repo_root>/artifacts/tests/coreclr/Android.x64.Release --copyNativeTestBin Release
--------------------------------------------------
```
To run all tests, execute that command, adding `Android` at the end.

1. Build and set the RuntimeFlavor
### Additional Documents
For more details about internals of the runtime tests, please refer to the [CoreCLR testing documents](../coreclr)

```bash
./build.sh /p:RuntimeFlavor=mono
## Libraries tests
### Desktop Mono
Build and run library tests against Mono JIT or interpreter
```
$(REPO_ROOT)/dotnet.sh build /t:Test /p:RuntimeFlavor=mono /p:Configuration=<Release/Debug> $(REPO_ROOT)/src/libraries/<library>/tests
```
or on Windows
```bat
build.cmd /p:RuntimeFlavor=mono
Alternatively, you could execute the following command from `$(REPO_ROOT)/src/mono/netcore`
```
make run-tests-corefx-<library>
```
For example, the following command is for running System.Runtime tests:
```
make run-tests-corefx-System.Runtime
```
### Mobile targets and WebAssembly
Build and run library tests against Webassembly, Android or iOS. See instructions located in [Library testing document folder](../libraries/)

2. cd into the test library of your choice (`cd src/libraries/<library>/tests`)
# Running the Mono samples
There are a few convenient samples located in `$(REPO_ROOT)/src/mono/netcore/sample`, which could help you test your program easily with different flavors of Mono or do a sanity check on the build. The samples are set up to work with a specific configuration; please refer to the relevant Makefile for specifics. If you would like to work with a different configuration, you can edit the Makefile.

3. Run the tests
## Desktop Mono
To run the desktop Mono sample, cd to `HelloWorld` and execute:

```
dotnet build /t:Test /p:RuntimeFlavor=mono
make run
```
Note that the default configuration of this sample is LLVM JIT.

# Patching Local dotnet (.dotnet-mono)
Another way to test mono out is by 'patching' a local dotnet with our runtime bits. This is a good way to write simple
test programs and get a glimpse of how mono will work with the dotnet tooling.
## WebAssembly
To run the WebAssembly sample, cd to `wasm`. There are two sub-folders `browser` and `console`. One is set up to run the progam in browser, the other is set up to run the program in console. Enter the desirable sub-folder and execute

```
make build && make run
```

To generate a local .dotnet-mono, execute this command:
## Android
To run the Android sample, cd to `Android` and execute

```
dotnet build /t:PatchLocalMonoDotnet $(REPO_ROOT)/src/mono/mono.proj
make run
```

You can then, for example, run our HelloWorld sample via:
## iOS
To run the iOS sample, cd to `iOS` and execute

```
dotnet build -c Release $(REPO_ROOT)/src/mono/netcore/sample/HelloWorld
MONO_ENV_OPTIONS="" COMPlus_DebugWriteToStdErr=1 \
$(REPO_ROOT)/.dotnet-mono/dotnet $(REPO_ROOT)/src/mono/netcore/sample/HelloWorld/bin/HelloWorld.dll
make run
```
113 changes: 110 additions & 3 deletions eng/Configurations.props
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,11 @@
steps and in the repository.
-->

<PropertyGroup>
<!-- Default to portable build if not explicitly set -->
<PortableBuild Condition="'$(PortableBuild)' == ''">true</PortableBuild>
</PropertyGroup>

<PropertyGroup>
<LibrariesProjectRoot>$([MSBuild]::NormalizeDirectory('$(RepoRoot)', 'src', 'libraries'))</LibrariesProjectRoot>
<CoreClrProjectRoot>$([MSBuild]::NormalizeDirectory('$(RepoRoot)', 'src', 'coreclr'))</CoreClrProjectRoot>
Expand Down Expand Up @@ -49,9 +54,6 @@
</PropertyGroup>

<PropertyGroup>
<HostRuntimeIdentifier Condition="'$(HostRuntimeIdentifier)' == '' and '$(MSBuildRuntimeType)' == 'core'">$([System.Runtime.InteropServices.RuntimeInformation]::RuntimeIdentifier)</HostRuntimeIdentifier>
<HostRuntimeIdentifier Condition="'$(HostRuntimeIdentifier)' == '' and '$(MSBuildRuntimeType)' != 'core'">win-$([System.Runtime.InteropServices.RuntimeInformation]::OSArchitecture.ToString().ToLowerInvariant)</HostRuntimeIdentifier>

<TargetOS Condition="'$(TargetOS)' == '' and $([MSBuild]::IsOSPlatform('OSX'))">OSX</TargetOS>
<TargetOS Condition="'$(TargetOS)' == '' and $([MSBuild]::IsOSPlatform('FREEBSD'))">FreeBSD</TargetOS>
<TargetOS Condition="'$(TargetOS)' == '' and $([MSBuild]::IsOSPlatform('NETBSD'))">NetBSD</TargetOS>
Expand All @@ -68,6 +70,109 @@
<TargetsMobile Condition="'$(TargetOS)' == 'iOS' or '$(TargetOS)' == 'Android' or '$(TargetOS)' == 'tvOS' or '$(TargetOS)' == 'Browser'">true</TargetsMobile>
</PropertyGroup>

<PropertyGroup Label="CalculateOS">
<_runtimeOS>$(RuntimeOS)</_runtimeOS>

<_parseDistroRid>$(__DistroRid)</_parseDistroRid>
<_parseDistroRid Condition="'$(_parseDistroRid)' == '' and '$(MSBuildRuntimeType)' == 'core'">$([System.Runtime.InteropServices.RuntimeInformation]::RuntimeIdentifier)</_parseDistroRid>
<_parseDistroRid Condition="'$(_parseDistroRid)' == '' and '$(MSBuildRuntimeType)' != 'core'">win-$([System.Runtime.InteropServices.RuntimeInformation]::OSArchitecture.ToString().ToLowerInvariant)</_parseDistroRid>
<_distroRidIndex>$(_parseDistroRid.LastIndexOfAny("-"))</_distroRidIndex>

<_runtimeOS Condition="'$(_runtimeOS)' == ''">$(_parseDistroRid.SubString(0, $(_distroRidIndex)))</_runtimeOS>

<!-- _runtimeOS is calculated based on the build system OS, however if building for Browser/iOS/Android we need to let
the build system to use browser/ios/android as the _runtimeOS for produced package RIDs. -->
<_runtimeOS Condition="'$(TargetsMobile)' == 'true'">$(TargetOS.ToLowerInvariant())</_runtimeOS>

<_runtimeOSVersionIndex>$(_runtimeOS.IndexOfAny(".-0123456789"))</_runtimeOSVersionIndex>
<_runtimeOSFamily Condition="'$(_runtimeOSVersionIndex)' != '-1'">$(_runtimeOS.SubString(0, $(_runtimeOSVersionIndex)))</_runtimeOSFamily>

<_buildingInOSX>$([MSBuild]::IsOSPlatform('OSX'))</_buildingInOSX>
<_portableOS>linux</_portableOS>
<_portableOS Condition="'$(_runtimeOS)' == 'linux-musl'">linux-musl</_portableOS>
<_portableOS Condition="$(_buildingInOSX)">osx</_portableOS>
<_portableOS Condition="'$(_runtimeOSFamily)' == 'win' or '$(TargetOS)' == 'windows'">win</_portableOS>
<_portableOS Condition="'$(_runtimeOSFamily)' == 'FreeBSD'">freebsd</_portableOS>
<_portableOS Condition="'$(_runtimeOSFamily)' == 'illumos'">illumos</_portableOS>
<_portableOS Condition="'$(_runtimeOSFamily)' == 'Solaris'">solaris</_portableOS>
<_portableOS Condition="'$(_runtimeOS)' == 'Browser'">browser</_portableOS>
<_portableOS Condition="'$(_runtimeOS)' == 'ios'">ios</_portableOS>
<_portableOS Condition="'$(_runtimeOS)' == 'tvos'">tvos</_portableOS>
<_portableOS Condition="'$(_runtimeOS)' == 'android'">android</_portableOS>

<_runtimeOS Condition="$(_runtimeOS.StartsWith('tizen'))">linux</_runtimeOS>
<_runtimeOS Condition="'$(PortableBuild)' == 'true'">$(_portableOS)</_runtimeOS>

<!-- support cross-targeting by choosing a RID to restore when running on a different machine that what we're build for -->
<_portableOS Condition="'$(TargetOS)' == 'Unix' and '$(_runtimeOSFamily)' != 'osx' and '$(_runtimeOSFamily)' != 'FreeBSD' and '$(_runtimeOS)' != 'linux-musl' and '$(_runtimeOSFamily)' != 'illumos' and '$(_runtimeOSFamily)' != 'Solaris'">linux</_portableOS>
</PropertyGroup>

<PropertyGroup Label="CalculateArch">
<_hostArch>$([System.Runtime.InteropServices.RuntimeInformation]::ProcessArchitecture.ToString().ToLowerInvariant)</_hostArch>
<TargetArchitecture Condition="'$(TargetArchitecture)' == '' and '$(_hostArch)' == 'arm'">arm</TargetArchitecture>
<TargetArchitecture Condition="'$(TargetArchitecture)' == '' and '$(_hostArch)' == 'arm64'">arm64</TargetArchitecture>
<TargetArchitecture Condition="'$(TargetArchitecture)' == '' and '$(TargetOS)' == 'Browser'">wasm</TargetArchitecture>
<TargetArchitecture Condition="'$(TargetArchitecture)' == '' and '$(TargetsMobile)' == 'true'">x64</TargetArchitecture>
<TargetArchitecture Condition="'$(TargetArchitecture)' == ''">x64</TargetArchitecture>
</PropertyGroup>

<PropertyGroup Label="CalculateRID">
<_toolRuntimeRID Condition="'$(BuildingInsideVisualStudio)' == 'true'">$(_runtimeOS)-x64</_toolRuntimeRID>
<_toolRuntimeRID Condition="'$(_toolRuntimeRID)' == ''">$(_runtimeOS)-$(_hostArch)</_toolRuntimeRID>
<!-- We build linux-musl-arm on a ubuntu container, so we can't use the toolset build for alpine runtime. We need to use portable linux RID for our toolset in order to be able to use it. -->
<_toolRuntimeRID Condition="'$(_runtimeOS)' == 'linux-musl' and $(TargetArchitecture.StartsWith('arm')) and !$(_hostArch.StartsWith('arm'))">linux-x64</_toolRuntimeRID>

<!-- There are no WebAssembly tools, so use the default ones -->
<_toolRuntimeRID Condition="'$(_runtimeOS)' == 'browser' and '$(TargetOS)' == 'windows'">win-x64</_toolRuntimeRID>
<_toolRuntimeRID Condition="'$(_runtimeOS)' == 'browser' and '$(TargetOS)' != 'windows' and $(_buildingInOSX)">osx-x64</_toolRuntimeRID>
<_toolRuntimeRID Condition="'$(_runtimeOS)' == 'browser' and '$(TargetOS)' != 'windows' and !$(_buildingInOSX)">linux-x64</_toolRuntimeRID>

<!-- There are no Android tools, so use the default ones -->
<_toolRuntimeRID Condition="'$(_runtimeOS)' == 'android' and '$(TargetOS)' == 'windows'">win-x64</_toolRuntimeRID>
<_toolRuntimeRID Condition="'$(_runtimeOS)' == 'android' and '$(TargetOS)' != 'windows' and $(_buildingInOSX)">osx-x64</_toolRuntimeRID>
<_toolRuntimeRID Condition="'$(_runtimeOS)' == 'android' and '$(TargetOS)' != 'windows' and !$(_buildingInOSX)">linux-x64</_toolRuntimeRID>

<!-- There are no iOS or tvOS tools and it can be built on OSX only, so use that -->
<_toolRuntimeRID Condition="'$(_runtimeOS)' == 'ios' or '$(_runtimeOS)' == 'tvos'">osx-x64</_toolRuntimeRID>
<MicrosoftNetCoreIlasmPackageRuntimeId>$(_toolRuntimeRID)</MicrosoftNetCoreIlasmPackageRuntimeId>

<_packageRID Condition="'$(PortableBuild)' == 'true'">$(_portableOS)-$(TargetArchitecture)</_packageRID>
<PackageRID Condition="'$(PackageRID)' == ''">$(_packageRID)</PackageRID>
<PackageRID Condition="'$(PackageRID)' == ''">$(_runtimeOS)-$(TargetArchitecture)</PackageRID>

<_outputRID Condition="'$(TargetOS)' == 'windows'">win-$(TargetArchitecture)</_outputRID>
<_outputRID Condition="'$(TargetOS)' == 'OSX'">osx-$(TargetArchitecture)</_outputRID>
<_outputRID Condition="'$(TargetOS)' == 'Linux'">linux-$(TargetArchitecture)</_outputRID>
<_outputRID Condition="'$(TargetOS)' == 'FreeBSD'">freebsd-$(TargetArchitecture)</_outputRID>
<_outputRID Condition="'$(TargetOS)' == 'NetBSD'">netbsd-$(TargetArchitecture)</_outputRID>
<_outputRID Condition="'$(TargetOS)' == 'illumos'">illumos-$(TargetArchitecture)</_outputRID>
<_outputRID Condition="'$(TargetOS)' == 'Solaris'">solaris-$(TargetArchitecture)</_outputRID>
<_outputRID Condition="'$(TargetOS)' == 'iOS'">ios-$(TargetArchitecture)</_outputRID>
<_outputRID Condition="'$(TargetOS)' == 'tvOS'">tvos-$(TargetArchitecture)</_outputRID>
<_outputRID Condition="'$(TargetOS)' == 'Android'">android-$(TargetArchitecture)</_outputRID>
<_outputRID Condition="'$(TargetOS)' == 'Browser'">browser-$(TargetArchitecture)</_outputRID>

<OutputRid Condition="'$(OutputRid)' == ''">$(PackageRID)</OutputRid>
<OutputRid Condition="'$(PortableBuild)' == 'true'">$(_outputRID)</OutputRid>
</PropertyGroup>

<PropertyGroup Label="CalculateTargetOSName" Condition="'$(SkipInferTargetOSName)' != 'true'">
<TargetsFreeBSD Condition="'$(TargetOS)' == 'FreeBSD'">true</TargetsFreeBSD>
<Targetsillumos Condition="'$(TargetOS)' == 'illumos'">true</Targetsillumos>
<TargetsSolaris Condition="'$(TargetOS)' == 'Solaris'">true</TargetsSolaris>
<TargetsLinux Condition="'$(TargetOS)' == 'Linux' or '$(TargetOS)' == 'Android'">true</TargetsLinux>
<TargetsNetBSD Condition="'$(TargetOS)' == 'NetBSD'">true</TargetsNetBSD>
<TargetsOSX Condition="'$(TargetOS)' == 'OSX'">true</TargetsOSX>
<TargetsiOS Condition="'$(TargetOS)' == 'iOS'">true</TargetsiOS>
<TargetstvOS Condition="'$(TargetOS)' == 'tvOS'">true</TargetstvOS>
<TargetsiOSSimulator Condition="'$(TargetsiOS)' == 'true' and ('$(TargetArchitecture)' == 'x64' or '$(TargetArchitecture)' == 'x86')">true</TargetsiOSSimulator>
<TargetstvOSSimulator Condition="'$(TargetstvOS)' == 'true' and '$(TargetArchitecture)' == 'x64'">true</TargetstvOSSimulator>
<TargetsAndroid Condition="'$(TargetOS)' == 'Android'">true</TargetsAndroid>
<TargetsBrowser Condition="'$(TargetOS)' == 'Browser'">true</TargetsBrowser>
<TargetsWindows Condition="'$(TargetOS)' == 'windows'">true</TargetsWindows>
<TargetsUnix Condition="'$(TargetsFreeBSD)' == 'true' or '$(Targetsillumos)' == 'true' or '$(TargetsSolaris)' == 'true' or '$(TargetsLinux)' == 'true' or '$(TargetsNetBSD)' == 'true' or '$(TargetsOSX)' == 'true' or '$(TargetstvOS)' == 'true' or '$(TargetsiOS)' == 'true' or '$(TargetsAndroid)' == 'true'">true</TargetsUnix>
</PropertyGroup>

<!--Feature switches -->
<PropertyGroup>
<EnableNgenOptimization Condition="'$(EnableNgenOptimization)' == '' and '$(Configuration)' == 'Release'">true</EnableNgenOptimization>
Expand All @@ -76,4 +181,6 @@
<!-- Turn off end of life target framework checks as we intentionally build older .NETCoreApp configurations. -->
<CheckEolTargetFramework>false</CheckEolTargetFramework>
</PropertyGroup>

<Import Project="$(MSBuildThisFileDirectory)native\naming.props" />
</Project>
Loading

0 comments on commit 7faa1f0

Please sign in to comment.