Skip to content

Commit

Permalink
[tests, build] multi-target Xamarin.Android.Build.Tests
Browse files Browse the repository at this point in the history
There are various tests in `Xamarin.Android.Build.Tests` that execute
MSBuild tasks directly. This means the changes in dotnet#4820 are not quite
complete. We need to actually *run* the tests under .NET Core to see
if these tests would pass under a One .NET context.

The solution (you would think), would be to make the test assembly
`netstandard2.0` so you could run either:

    > nunit3-console Xamarin.Android.Build.Tests.dll
    > dotnet test Xamarin.Android.Build.Tests.dll

`nunit3-console` would execute the tests under .NET framework/Mono and
`dotnet test` would run under .NET Core or .NET 5+.

Unfortunately, `nunit3-console` can't run tests from a
`netstandard2.0` library!

So we multi-target:

    <TargetFrameworks>net472;net5.0</TargetFrameworks>

Unfortunately, the `net5.0` causes MSBuild to fail with:

    > msbuild Xamarin.Android.sln
    ...
    (GetReferenceAssemblyPaths target) ->
      C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\Current\Bin\Microsoft.Common.CurrentVersion.targets(1177,5): error MSB3644: The reference assemblies for .NETFramework,Version=v5.0 were not found. To resolve this, install the Developer Pack (SDK/Targeting Pack) for this framework version or retarget your application. You can download .NET Framework Developer Packs at https://aka.ms/msbuild/developerpacks

I think there is some expectation to use `dotnet build` for .NET 5
instead. We don't want to move our *entire* build over to `dotnet
build` yet (we probably can't!), so instead we can do:

    <TargetFrameworks>net472;netcoreapp3.1</TargetFrameworks>

This allows `msbuild Xamarin.Android.sln` to continue to work. This
might also be something that gets fixed in a future MSBuild version.

I updated various `.yaml` definitions so we can run NUnit tests via
`nunit3-console` or `dotnet test` as needed.

Other things that broke here:

* We need `Microsoft.NET.Test.Sdk` for `dotnet test` to work.
* `Builder.UseDotNet` can now be set based on a `#if` instead of an
  NUnit parameter.
* `Xamarin.Android.Tools.Aidl` needs to be `netstandard2.0`. This
  required including a `System.CodeDom` package.
* `Xamarin.Android.Build.Tasks` can't use `FSharp.Core` in
  `netcoreapp3.1`. I just removed it, since it is not used.
* Updated various paths to `Xamarin.Android.Build.Tests.dll`.
  • Loading branch information
jonathanpeppers committed Aug 10, 2020
1 parent de70822 commit 2f65303
Show file tree
Hide file tree
Showing 12 changed files with 54 additions and 60 deletions.
2 changes: 1 addition & 1 deletion .vscode/launch.json
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
"name": "Launch",
"type": "mono",
"request": "launch",
"program": "${workspaceRoot}packages/nunit.consolerunner/3.11.1/tools/nunit3-console.exe ${workspaceRoot}bin/TestDebug/Xamarin.Android.Build.Tests.dll",
"program": "${workspaceRoot}packages/nunit.consolerunner/3.11.1/tools/nunit3-console.exe ${workspaceRoot}bin/TestDebug/net472/Xamarin.Android.Build.Tests.dll",
"cwd": "${workspaceRoot}bin/TestDebug/"
},
{
Expand Down
2 changes: 1 addition & 1 deletion .vscode/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"nxunitExplorer.nunit": "packages/nunit.consolerunner/3.11.1/tools/nunit3-console.exe",
"nxunitExplorer.modules": [
"bin/TestDebug/MSBuildDeviceIntegration/MSBuildDeviceIntegration.dll",
"bin/TestDebug/Xamarin.Android.Build.Tests.dll",
"bin/TestDebug/net472/Xamarin.Android.Build.Tests.dll",
"bin/TestDebug/Xamarin.Android.Build.Tests.Commercial.dll",
],
"cmake.configureOnOpen": false
Expand Down
16 changes: 8 additions & 8 deletions build-tools/automation/azure-pipelines.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ variables:
# - This is a non-fork branch with name containing "mono-" (for Mono bumps)
IsMonoBranch: $[and(eq(variables['XA.Commercial.Build'], 'true'), ne(variables['System.PullRequest.IsFork'], 'True'), or(contains(variables['Build.SourceBranchName'], 'mono-'), contains(variables['System.PullRequest.SourceBranch'], 'mono-')))]
RunAllTests: $[or(eq(variables['XA.RunAllTests'], true), eq(variables['IsMonoBranch'], true))]
DotNetNUnitCategories: '&& cat != DotNetIgnore && cat != AOT && cat != FSharp && cat != LibraryProjectZip && cat != MkBundle && cat != MonoSymbolicate && cat != PackagesConfig && cat != StaticProject'
DotNetNUnitCategories: '&& TestCategory != DotNetIgnore && TestCategory != AOT && TestCategory != FSharp && TestCategory != LibraryProjectZip && TestCategory != MkBundle && TestCategory != MonoSymbolicate && TestCategory != PackagesConfig && TestCategory != StaticProject'

# Stage and Job "display names" are shortened because they are combined to form the name of the corresponding GitHub check.
stages:
Expand Down Expand Up @@ -366,7 +366,7 @@ stages:
- template: yaml-templates\run-nunit-tests.yaml
parameters:
testRunTitle: Smoke MSBuild Tests - Windows Build Tree
testAssembly: $(System.DefaultWorkingDirectory)\bin\Test$(XA.Build.Configuration)\Xamarin.Android.Build.Tests.dll
testAssembly: $(System.DefaultWorkingDirectory)\bin\Test$(XA.Build.Configuration)\net472\Xamarin.Android.Build.Tests.dll
testResultsFile: TestResult-SmokeMSBuildTests-WinBuildTree-$(XA.Build.Configuration).xml
nunitConsoleExtraArgs: --where "cat == SmokeTests" --workers=4

Expand Down Expand Up @@ -1054,23 +1054,23 @@ stages:
job_name: mac_dotnet_tests_1
job_suffix: One .NET
nunit_categories: $(DotNetNUnitCategories)
nunit_extra: --testparam dotnet=true
target_framework: 'netcoreapp3.1'

- template: yaml-templates\run-msbuild-mac-tests.yaml
parameters:
node_id: 2
job_name: mac_dotnet_tests_2
job_suffix: One .NET
nunit_categories: $(DotNetNUnitCategories)
nunit_extra: --testparam dotnet=true
target_framework: 'netcoreapp3.1'

- template: yaml-templates\run-msbuild-mac-tests.yaml
parameters:
node_id: 3
job_name: mac_dotnet_tests_3
job_suffix: One .NET
nunit_categories: $(DotNetNUnitCategories)
nunit_extra: --testparam dotnet=true
target_framework: 'netcoreapp3.1'

# Xamarin.Android (Test MSBuild - Windows - One .NET)
- template: yaml-templates\run-msbuild-win-tests.yaml
Expand All @@ -1079,23 +1079,23 @@ stages:
job_name: win_dotnet_tests_1
job_suffix: One .NET
nunit_categories: $(DotNetNUnitCategories)
nunit_extra: --testparam dotnet=true
target_framework: 'netcoreapp3.1'

- template: yaml-templates\run-msbuild-win-tests.yaml
parameters:
node_id: 2
job_name: win_dotnet_tests_2
job_suffix: One .NET
nunit_categories: $(DotNetNUnitCategories)
nunit_extra: --testparam dotnet=true
target_framework: 'netcoreapp3.1'

- template: yaml-templates\run-msbuild-win-tests.yaml
parameters:
node_id: 3
job_name: win_dotnet_tests_3
job_suffix: One .NET
nunit_categories: $(DotNetNUnitCategories)
nunit_extra: --testparam dotnet=true
target_framework: 'netcoreapp3.1'

- stage: integrated_regression_test
displayName: Regression Tests
Expand Down
15 changes: 11 additions & 4 deletions build-tools/automation/yaml-templates/run-msbuild-mac-tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ parameters:
job_name: ''
job_suffix: ''
nunit_categories: ''
nunit_extra: ''
target_framework: 'net472'
run_extra_tests: false

jobs:
Expand All @@ -16,6 +16,8 @@ jobs:
cancelTimeoutInMinutes: 5
workspace:
clean: all
variables:
UseDotNet: ${{ eq(parameters.target_framework, 'netcoreapp3.1') }}
steps:
- template: setup-test-environment.yaml

Expand All @@ -31,24 +33,29 @@ jobs:

- template: run-nunit-tests.yaml
parameters:
useDotNet: $(UseDotNet)
testRunTitle: Xamarin.Android.Build.Tests - macOS-${{ parameters.node_id }} - ${{ parameters.job_suffix }}
testAssembly: $(System.DefaultWorkingDirectory)/bin/Test$(XA.Build.Configuration)/Xamarin.Android.Build.Tests.dll
nunitConsoleExtraArgs: --where "cat == Node-${{ parameters.node_id }} ${{ parameters.nunit_categories }}" ${{ parameters.nunit_extra }}
testAssembly: $(System.DefaultWorkingDirectory)/bin/Test$(XA.Build.Configuration)/${{ parameters.target_framework }}/Xamarin.Android.Build.Tests.dll
nunitConsoleExtraArgs: --where "cat == Node-${{ parameters.node_id }} ${{ parameters.nunit_categories }}"
dotNetTestExtraArgs: --filter "TestCategory = Node-${{ parameters.node_id }} ${{ parameters.nunit_categories }}"
testResultsFile: TestResult-MSBuildTests-${{ parameters.job_name }}-$(XA.Build.Configuration).xml

# Only run these tests on node 2
- ${{ if eq(parameters.run_extra_tests, true) }}:
- template: run-nunit-tests.yaml
parameters:
useDotNet: $(UseDotNet)
testRunTitle: Xamarin.Android.Build.Tests.Commercial - macOS
testAssembly: $(System.DefaultWorkingDirectory)/bin/Test$(XA.Build.Configuration)/Xamarin.Android.Build.Tests.Commercial.dll
testResultsFile: TestResult-MSBuildTestsCommercial-macOS-$(XA.Build.Configuration).xml

- template: run-nunit-tests.yaml
parameters:
useDotNet: $(UseDotNet)
testRunTitle: Xamarin.Android.Build.Tests - macOS - No Node
testAssembly: $(System.DefaultWorkingDirectory)/bin/Test$(XA.Build.Configuration)/Xamarin.Android.Build.Tests.dll
testAssembly: $(System.DefaultWorkingDirectory)/bin/Test$(XA.Build.Configuration)/${{ parameters.target_framework }}/Xamarin.Android.Build.Tests.dll
nunitConsoleExtraArgs: --where "cat != Node-1 && cat != Node-2 && cat != Node-3"
dotNetTestExtraArgs: --filter "TestCategory != Node-1 && TestCategory != Node-2 && TestCategory != Node-3"
testResultsFile: TestResult-MSBuildTests-macOS-NoNode-$(XA.Build.Configuration).xml

- template: upload-results.yaml
Expand Down
14 changes: 10 additions & 4 deletions build-tools/automation/yaml-templates/run-msbuild-win-tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ parameters:
job_name: ''
job_suffix: ''
nunit_categories: ''
nunit_extra: ''
target_framework: 'net472'
run_extra_tests: false

jobs:
Expand All @@ -16,6 +16,7 @@ jobs:
cancelTimeoutInMinutes: 5
variables:
VSINSTALLDIR: C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise
UseDotNet: ${{ eq(parameters.target_framework, 'netcoreapp3.1') }}
steps:

- template: kill-processes.yaml
Expand All @@ -42,24 +43,29 @@ jobs:
# Using all available cores seems to occasionally bog down our machines and cause parallel test execution to slow down dramatically.
- template: run-nunit-tests.yaml
parameters:
useDotNet: $(UseDotNet)
testRunTitle: Xamarin.Android.Build.Tests - Windows-${{ parameters.node_id }} - ${{ parameters.job_suffix }}
testAssembly: $(System.DefaultWorkingDirectory)\bin\Test$(XA.Build.Configuration)\Xamarin.Android.Build.Tests.dll
nunitConsoleExtraArgs: --workers=4 --where "cat == Node-${{ parameters.node_id }} ${{ parameters.nunit_categories }}" ${{ parameters.nunit_extra }}
testAssembly: $(System.DefaultWorkingDirectory)\bin\Test$(XA.Build.Configuration)\${{ parameters.target_framework }}\Xamarin.Android.Build.Tests.dll
nunitConsoleExtraArgs: --workers=4 --where "cat == Node-${{ parameters.node_id }} ${{ parameters.nunit_categories }}"
dotNetTestExtraArgs: --filter "TestCategory = Node-${{ parameters.node_id }} ${{ parameters.nunit_categories }}"
testResultsFile: TestResult-MSBuildTests-${{ parameters.job_name }}-$(XA.Build.Configuration).xml

# Only run these tests on node 2
- ${{ if eq(parameters.run_extra_tests, true) }}:
- template: run-nunit-tests.yaml
parameters:
useDotNet: $(UseDotNet)
testRunTitle: Xamarin.Android.Build.Tests.Commercial - Windows
testAssembly: $(System.DefaultWorkingDirectory)\bin\Test$(XA.Build.Configuration)\Xamarin.Android.Build.Tests.Commercial.dll
testResultsFile: TestResult-MSBuildTestsCommercial-Windows-$(XA.Build.Configuration).xml

- template: run-nunit-tests.yaml
parameters:
useDotNet: $(UseDotNet)
testRunTitle: Xamarin.Android.Build.Tests - Windows - No Node
testAssembly: $(System.DefaultWorkingDirectory)\bin\Test$(XA.Build.Configuration)\Xamarin.Android.Build.Tests.dll
testAssembly: $(System.DefaultWorkingDirectory)\bin\Test$(XA.Build.Configuration)\${{ parameters.target_framework }}\Xamarin.Android.Build.Tests.dll
nunitConsoleExtraArgs: --workers=4 --where "cat != Node-1 && cat != Node-2 && cat != Node-3"
dotNetTestExtraArgs: --filter "TestCategory != Node-1 && TestCategory != Node-2 && TestCategory != Node-3"
testResultsFile: TestResult-MSBuildTests-Windows-NoNode-$(XA.Build.Configuration).xml

- template: upload-results.yaml
Expand Down
6 changes: 5 additions & 1 deletion build-tools/automation/yaml-templates/run-nunit-tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,15 @@ parameters:
testResultsFile: TestResult.xml
testResultsFormat: NUnit
nunitConsoleExtraArgs: ''
dotNetTestExtraArgs: ''
useDotNet: false
condition: succeeded()

steps:
- powershell: |
if ([Environment]::OSVersion.Platform -eq "Unix") {
if ("${{ parameters.useDotNet }}" -eq $true) {
dotnet test ${{ parameters.testAssembly }} --logger "trx;LogFileName=${{ parameters.testResultsFile }}" ${{ parameters.dotNetTestExtraArgs }}
} elseif ([Environment]::OSVersion.Platform -eq "Unix") {
mono ${{ parameters.nunitConsole }} ${{ parameters.testAssembly }} --result ${{ parameters.testResultsFile }} ${{ parameters.nunitConsoleExtraArgs }}
} else {
${{ parameters.nunitConsole }} ${{ parameters.testAssembly }} --result ${{ parameters.testResultsFile }} ${{ parameters.nunitConsoleExtraArgs }}
Expand Down
1 change: 0 additions & 1 deletion build-tools/installers/create-installers.targets
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,6 @@
<_MSBuildFiles Include="$(MSBuildSrcDir)\cil-strip.pdb" Condition=" '$(PackageId)' != 'Microsoft.Android.Sdk' " />
<_MSBuildFiles Include="$(MSBuildSrcDir)\class-parse.exe" Condition=" '$(PackageId)' != 'Microsoft.Android.Sdk' " />
<_MSBuildFiles Include="$(MSBuildSrcDir)\class-parse.pdb" Condition=" '$(PackageId)' != 'Microsoft.Android.Sdk' " />
<_MSBuildFiles Include="$(MSBuildSrcDir)\FSharp.Core.dll" />
<_MSBuildFiles Include="$(MSBuildSrcDir)\generator.exe" Condition=" '$(PackageId)' != 'Microsoft.Android.Sdk' " />
<_MSBuildFiles Include="$(MSBuildSrcDir)\generator.pdb" Condition=" '$(PackageId)' != 'Microsoft.Android.Sdk' " />
<_MSBuildFiles Include="$(MSBuildSrcDir)\HtmlAgilityPack.dll" />
Expand Down
2 changes: 1 addition & 1 deletion build-tools/scripts/RunTests.targets
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
<_XABuildProperties>$(_XABuildDiag)/p:Configuration=$(Configuration)</_XABuildProperties>
</PropertyGroup>
<ItemGroup>
<_TestAssembly Include="$(_TopDir)\bin\Test$(Configuration)\Xamarin.Android.Build.Tests.dll" />
<_TestAssembly Include="$(_TopDir)\bin\Test$(Configuration)\net472\Xamarin.Android.Build.Tests.dll" />
<_ApkTestProject Include="$(_TopDir)\src\Mono.Android\Test\Mono.Android-Tests.csproj" />
<_ApkTestProject Include="$(_TopDir)\tests\CodeGen-Binding\Xamarin.Android.JcwGen-Tests\Xamarin.Android.JcwGen-Tests.csproj" />
<_ApkTestProject Include="$(_TopDir)\tests\CodeGen-MkBundle\Xamarin.Android.MakeBundle-Tests\Xamarin.Android.MakeBundle-Tests.csproj" />
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,11 @@ public static string AndroidMSBuildDirectory {

static SetUp ()
{
Builder.UseDotNet = string.Equals (TestContext.Parameters ["dotnet"], bool.TrueString, StringComparison.OrdinalIgnoreCase);
#if NETCOREAPP3_1
Builder.UseDotNet = true;
#else
Builder.UseDotNet = false;
#endif

using (var builder = new Builder ()) {
CommercialBuildAvailable = File.Exists (Path.Combine (builder.AndroidMSBuildDirectory, "Xamarin.Android.Common.Debugging.targets"));
Expand Down
Original file line number Diff line number Diff line change
@@ -1,9 +1,8 @@
<Project Sdk="Microsoft.NET.Sdk">

<PropertyGroup>
<TargetFramework>net472</TargetFramework>
<TargetFrameworks>net472;netcoreapp3.1</TargetFrameworks>
<LibZipSharpBundleAllNativeLibraries>true</LibZipSharpBundleAllNativeLibraries>
<AppendTargetFrameworkToOutputPath>false</AppendTargetFrameworkToOutputPath>
<OutputPath>..\..\..\..\bin\Test$(Configuration)</OutputPath>
</PropertyGroup>

Expand All @@ -17,6 +16,12 @@
</Reference>
</ItemGroup>

<!-- Required packages for .NET Core -->
<ItemGroup Condition=" '$(TargetFramework)' == 'netcoreapp3.1' ">
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.6.1" />
<PackageReference Include="System.Configuration.ConfigurationManager" Version="4.7.0" />
</ItemGroup>

<ItemGroup>
<ProjectReference Include="..\Xamarin.ProjectTools\Xamarin.ProjectTools.csproj" />
<ProjectReference Include="..\..\Xamarin.Android.Build.Tasks.csproj" />
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,6 @@
</ItemGroup>

<ItemGroup>
<PackageReference Include="FSharp.Core" Version="3.1.2.5" />
<PackageReference Include="ILRepack" Version="2.0.18" />
<PackageReference Include="Irony" Version="1.1.0" />
<PackageReference Include="Newtonsoft.Json" Version="12.0.3" />
Expand Down
40 changes: 5 additions & 35 deletions src/Xamarin.Android.Tools.Aidl/Xamarin.Android.Tools.Aidl.csproj
Original file line number Diff line number Diff line change
@@ -1,45 +1,15 @@
<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build" ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
<Configuration Condition=" '$(Configuration)' == '' ">Debug</Configuration>
<Platform Condition=" '$(Platform)' == '' ">AnyCPU</Platform>
<ProjectGuid>{D27AD8F7-7710-40BE-B03B-55EFBEC13C44}</ProjectGuid>
<OutputType>Library</OutputType>
<RootNamespace>Xamarin.AndroidTools.Aidl</RootNamespace>
<AssemblyName>Xamarin.Android.Tools.Aidl</AssemblyName>
<DebugSymbols>True</DebugSymbols>
</PropertyGroup>
<Project Sdk="Microsoft.NET.Sdk">
<Import Project="..\..\Configuration.props" />
<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
<Optimize>False</Optimize>
<OutputPath>$(XAInstallPrefix)xbuild\Xamarin\Android</OutputPath>
<DefineConstants>DEBUG;</DefineConstants>
<ErrorReport>prompt</ErrorReport>
<WarningLevel>4</WarningLevel>
<ConsolePause>False</ConsolePause>
</PropertyGroup>
<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|AnyCPU' ">
<Optimize>True</Optimize>
<PropertyGroup>
<TargetFramework>netstandard2.0</TargetFramework>
<AppendTargetFrameworkToOutputPath>false</AppendTargetFrameworkToOutputPath>
<OutputPath>$(XAInstallPrefix)xbuild\Xamarin\Android</OutputPath>
<ErrorReport>prompt</ErrorReport>
<WarningLevel>4</WarningLevel>
<ConsolePause>False</ConsolePause>
</PropertyGroup>
<ItemGroup>
<Reference Include="System" />
<Reference Include="System.Core" />
<Reference Include="Xamarin.Android.Cecil">
<HintPath>..\..\bin\$(Configuration)\lib\xamarin.android\xbuild\Xamarin\Android\Xamarin.Android.Cecil.dll</HintPath>
</Reference>
<PackageReference Include="Irony" Version="1.1.0" />
</ItemGroup>
<Import Project="$(MSBuildBinPath)\Microsoft.CSharp.targets" />
<ItemGroup>
<Compile Include="AidlAst.cs" />
<Compile Include="AidlCompiler.cs" />
<Compile Include="AidlParser.cs" />
<Compile Include="BindingDatabase.cs" />
<Compile Include="CSharpCodeGenerator.cs" />
<Compile Include="ConverterOptions.cs" />
<PackageReference Include="System.CodeDom" Version="4.7.0" />
</ItemGroup>
</Project>

0 comments on commit 2f65303

Please sign in to comment.