Run tests in parallel, one per framework (#354)

* Run tests in parallel, one per framework

Now that tests run on three frameworks (net461, net6.0, net8.0), we can
save a lot of build time by running each framework's tests in parallel.

* Ensure parallel test runs don't cancel each other

* Don't specify framework for `dotnet build` step

* Attempt to work around `dotnet test` bug

If running with `--no-build`, then `dotnet test` complains about DLLs
being invalid arguments, resulting in an exit code of 1 even if all
tests passed. This then makes GHA think the build was a failure. The
discussion in the dotnet SDK issue suggests removing `--no-build` as
one possible workaround for this issue.

* Try doing restore-build-test in one go on Linux

Otherwise we get errors about the dotnet restore not being done for the
right framework.

* Try restoring for specific target framework

* Try specifying platform on Linux

Also, forgot to specify target framework on Windows; no wonder it was
taking so much longer!

* Try not specifying runtime for Linux

* ChorusMerge tests should run on all frameworks

Since ChorusMerge runs on net461;net6.0;net8.0, the ChorusMerge tests
should be run on all the same frameworks.

* Explicitly specify test project per step

Also only run Chorus.Tests on net461, since it's not defined for net6.0
and net8.0.

* Correct path to LibChorus tests

* lowered NUnit3TestAdapter version to a version which supports net461

* run Chorus hub tests in CI

* Checkout sources before building installer

Now that the installer job is not running in the same job matrix as the
tests, we need to re-run the dotnet restore and build steps again.

* Remove net6.0 targets

Everything that uses Chorus now runs on net8.0 or higher.

---------

Co-authored-by: Kevin Hahn <kevin_hahn@sil.org>

.github/workflows/ci+cd.yml

@@ -26,8 +26,13 @@ jobs:
       fail-fast: false
       matrix:
         os: [ubuntu-latest, windows-latest]
+        framework: [net461, net8.0]
+        exclude:
+          # dotnet on Linux cannot build net461 without additional, unnecessary, work
+          - os: ubuntu-latest
+            framework: net461
     concurrency:
-      group: ${{ github.workflow }}-${{ github.ref }}-${{ matrix.os }}
+      group: ${{ github.workflow }}-${{ github.ref }}-${{ matrix.os }}-${{ matrix.framework }}
       cancel-in-progress: true
 
     steps:
@@ -53,44 +58,60 @@ jobs:
       run: sudo apt-get install python2
       if: matrix.os == 'ubuntu-latest'
 
-    # Unfortunately we need two test steps because we need different filters.
-    # We could conditionally set an environment variable, but unfortunately
-    # the syntax to access that is different on Windows vs Linux.
-    - name: Test on Linux
-      run: dotnet test --no-build -c Release --filter TestCategory!=RequiresUI -- NUnit.TestOutputXml=TestResults
-      if: matrix.os == 'ubuntu-latest'
+    - name: Test Chorus
+      run: dotnet test src/Chorus.Tests/Chorus.Tests.csproj -f ${{ matrix.framework }} --no-build -c Release --filter TestCategory!=SkipOnBuildServer -- NUnit.TestOutputXml=TestResults
+      if: matrix.framework == 'net461'
+
+    - name: Test Chorus Hub
+      run: dotnet test src/ChorusHubTests/ChorusHubTests.csproj -f ${{ matrix.framework }} --no-build -c Release --filter TestCategory!=SkipOnBuildServer -- NUnit.TestOutputXml=TestResults
+      if: matrix.framework == 'net461'
+
+    - name: Test ChorusMerge
+      run: dotnet test src/ChorusMerge.Tests/ChorusMerge.Tests.csproj -f ${{ matrix.framework }} --no-build -c Release --filter TestCategory!=SkipOnBuildServer -- NUnit.TestOutputXml=TestResults
 
-    - name: Test on Windows
-      run: dotnet test --no-build -c Release --filter TestCategory!=SkipOnBuildServer -- NUnit.TestOutputXml=TestResults
-      if: matrix.os == 'windows-latest'
+    - name: Test LibChorus
+      run: dotnet test src/LibChorusTests/LibChorus.Tests.csproj -f ${{ matrix.framework }} --no-build -c Release --filter TestCategory!=SkipOnBuildServer -- NUnit.TestOutputXml=TestResults
 
+    # TODO: Give each test result its own name based on OS and framework and change the test-results.yml workflow accordingly, so that they don't overwrite each other
     - name: Upload Test Results
       if: always()
       uses: actions/upload-artifact@v4
       with:
-        name: Test Results (${{matrix.os}})
+        name: Test Results (${{ matrix.framework }} on ${{matrix.os}})
         path: "**/TestResults/*.xml"
 
+  build-installers:
+    name: "Build Windows installers"
+    runs-on: windows-latest
+    needs: build-and-test
+    steps:
+    - name: Checkout
+      uses: actions/checkout@2541b1294d2704b0964813337f33b291d3f8596b # v3.0.2
+      with:
+        fetch-depth: 0 # fetch full history for GitVersion
+
+    - name: Restore
+      run: dotnet restore
+
+    - name: Build
+      run: dotnet build --no-restore -c Release
+
     - name: Checkout Chorus Help # required for Chorus Merge Module
       uses: actions/checkout@2541b1294d2704b0964813337f33b291d3f8596b # v3.0.2
       with:
         repository: sillsdev/chorushelp
         path: DistFiles/Help
-      if: matrix.os == 'windows-latest'
 
     - name: Add msbuild to PATH
       uses: microsoft/setup-msbuild@34cfbaee7f672c76950673338facd8a73f637506 # v1.1.3
-      if: matrix.os == 'windows-latest'
 
     # ChorusMergeModule.msm is used by FieldWorks, FLExBridge, and possibly WeSay installers (as of 2022.12).
     # CMM must be built after tests have been run, since the fixutf8.pyc files are generated only when the .py  files are run.
     - name: Build Merge Module
       run: msbuild src/Installer/ChorusMergeModule.wixproj
-      if: matrix.os == 'windows-latest'
 
     - name: Pack Merge Module
       run: msbuild src/Installer/ChorusMergeModule.wixproj -t:pack
-      if: matrix.os == 'windows-latest'
 
     - name: Upload Merge Module
       uses: actions/upload-artifact@v4
@@ -99,22 +120,19 @@ jobs:
         path: |
           output/Release/*.msm
           output/*.nupkg
-      if: matrix.os == 'windows-latest'
 
     - name: Build Chorus Hub Installer
       run: |
         msbuild src/Installer/ChorusHub.wixproj /t:Restore
         msbuild src/Installer/ChorusHub.wixproj /t:StampLicenseYear
         msbuild src/Installer/ChorusHub.wixproj /t:Build
-      if: matrix.os == 'windows-latest'
 
     - name: Upload Chorus Hub Installer
       uses: actions/upload-artifact@v4
       with:
         name: chorus-hub-installer
         path: |
           output/Release/*.msi
-      if: matrix.os == 'windows-latest'
 
     - name: Pack
       run: dotnet pack --no-restore --no-build -c Release
@@ -126,12 +144,11 @@ jobs:
         path: |
           output/*.nupkg
           output/*.snupkg
-      if: matrix.os == 'ubuntu-latest'
 
   publish-nuget:
     name: "Publish NuGet package"
     runs-on: ubuntu-latest
-    needs: build-and-test
+    needs: build-installers
     if: github.event_name == 'push'
     steps:
     - name: Download Artifacts

src/Chorus.Tests/Chorus.Tests.csproj

@@ -13,7 +13,7 @@
     <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.3.1" />
     <PackageReference Include="Microsoft.SourceLink.GitHub" Version="1.1.1" PrivateAssets="All" />
     <PackageReference Include="NUnit" Version="3.13.3" />
-    <PackageReference Include="NUnit3TestAdapter" Version="4.4.2" />
+    <PackageReference Include="NUnit3TestAdapter" Version="4.3.2" />
   </ItemGroup>
 
   <ItemGroup>

src/ChorusHubTests/ChorusHubTests.csproj

@@ -13,7 +13,7 @@
     <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.3.1" />
     <PackageReference Include="Microsoft.SourceLink.GitHub" Version="1.1.1" PrivateAssets="All" />
     <PackageReference Include="NUnit" Version="3.13.3" />
-    <PackageReference Include="NUnit3TestAdapter" Version="4.4.2" />
+    <PackageReference Include="NUnit3TestAdapter" Version="4.3.2"/>
   </ItemGroup>
 
   <ItemGroup>

src/ChorusMerge.Tests/ChorusMerge.Tests.csproj

@@ -4,6 +4,7 @@
     <RootNamespace>ChorusMerge.Tests</RootNamespace>
     <AssemblyTitle>ChorusMerge.Tests</AssemblyTitle>
     <Description>Unit tests for LibChorus.dll</Description>
+    <TargetFrameworks>net461;net8.0</TargetFrameworks>
     <IsTestProject>true</IsTestProject>
     <IsPackable>false</IsPackable>
   </PropertyGroup>
@@ -14,7 +15,7 @@
     <PackageReference Include="Microsoft.SourceLink.GitHub" Version="1.1.1" PrivateAssets="All" />
     <PackageReference Include="Newtonsoft.Json" Version="13.0.2" />
     <PackageReference Include="NUnit" Version="3.13.3" />
-    <PackageReference Include="NUnit3TestAdapter" Version="4.4.2" />
+    <PackageReference Include="NUnit3TestAdapter" Version="4.3.2"/>
     <PackageReference Include="SIL.TestUtilities" Version="12.0.0-*" />
   </ItemGroup>
 

src/ChorusMerge/ChorusMerge.csproj

@@ -5,7 +5,7 @@
     <AssemblyTitle>ChorusMerge</AssemblyTitle>
     <PackageId>SIL.Chorus.ChorusMerge</PackageId>
     <OutputType>Exe</OutputType>
-    <TargetFrameworks>net461;net6.0;net8.0</TargetFrameworks>
+    <TargetFrameworks>net461;net8.0</TargetFrameworks>
     <RepositoryUrl>https://github.com/sillsdev/chorus.git</RepositoryUrl>
     <AutoGenerateBindingRedirects>true</AutoGenerateBindingRedirects>
   </PropertyGroup>

src/LibChorusTests/LibChorus.Tests.csproj

@@ -4,7 +4,7 @@
     <RootNamespace>LibChorus.Tests</RootNamespace>
     <AssemblyTitle>LibChorus.Tests</AssemblyTitle>
     <Description>Unit tests for LibChorus.dll</Description>
-    <TargetFrameworks>net461;net6.0;net8.0</TargetFrameworks>
+    <TargetFrameworks>net461;net8.0</TargetFrameworks>
     <IsTestProject>true</IsTestProject>
     <IsPackable>false</IsPackable>
   </PropertyGroup>
@@ -15,7 +15,7 @@
     <PackageReference Include="Microsoft.SourceLink.GitHub" Version="1.1.1" PrivateAssets="All" />
     <PackageReference Include="Newtonsoft.Json" Version="13.0.2" />
     <PackageReference Include="NUnit" Version="3.13.3" />
-    <PackageReference Include="NUnit3TestAdapter" Version="4.4.2" />
+    <PackageReference Include="NUnit3TestAdapter" Version="4.3.2"/>
     <PackageReference Include="SharpZipLib" Version="1.3.3" />
     <PackageReference Include="SIL.Core" Version="12.0.0-*" />
     <PackageReference Include="SIL.TestUtilities" Version="12.0.0-*" />