Skip to content
This repository has been archived by the owner on Aug 8, 2024. It is now read-only.

Add support for building on and targeting Mono #50

Merged
merged 25 commits into from
Apr 23, 2018

Conversation

radical
Copy link
Member

@radical radical commented Apr 20, 2018

We used old netcore 1.x for building msbuild. We wanted to move to newer netcore 2.x for many reasons like being able to use PackageReference, switch to newer SDK projects and being able to build on High Sierra. Upstream master moved switched over to that for netcore and full framework builds. This PR adds support for using that new system to build for mono.

Steps to build are still the same:

build: make all-mono
test: make test-mono
clean: make clean

install: ./install-mono-prefix.sh <mono_prefix>

Also, we now try to watch for any changes in the build directory like new assemblies or files showing up or getting removed due to (for example) changes upstream. This way we always "approve" any such changes for our fork.
mono/README.md has a little more info on this.

@radical
Copy link
Member Author

radical commented Apr 20, 2018

Not ready for review yet. Pending:

  • Test with a real .pkg from the CI
  • 2 failing tests - probably need to file an upstream issue for these

@radical radical changed the title Xm mono port Add support for building on and targeting Mono Apr 20, 2018
radical added 8 commits April 20, 2018 20:22
This was breaking a test in Engine. The check is not required and now
matches what upstream has. Usage of
`ToolsetConfigurationReaderHelpers.ConfigurationFileMayHaveToolsets`
will get cleaned up when we update and push the target framework fallback
PR upstream.
.. that is what xbuild provided. For 15.0+ we setup the toolset from
app.config .
	`$ ./build.sh -host mono`

- Using TargetFramework=net461
- This builds but some tests still fail
Assembly cleanup failed because the environment had the
`MSBUILDTARGETOUTPUTLOGGING` environment variable removed. This was
because NodePackets_Tests.TestTranslation always removed that instead of
restoring it to it's original value.
    Microsoft.Build.UnitTests.UtilitiesTestStandard.CommentsInPreprocessing

This test depends on `-pp`/preprocessing and directly invokes
`MSBuildApp.Execute`, which sets the environment variable
`MSBUILDLOADALLFILESASWRITEABLE` and never unsets it.

`TestEnvironment`+`MSBuildTestEnvironmentFixture` collects the set of
environment variables before the test begins but finds the extra
`MSBUILDLOADALLFILESASWRITEABLE` at the end causing a failure in the
test cleanup. This cascades into a test-class cleanup failure and on to
an assembly level cleanup failure.

This shows up as xunit `Error` on netcore and as a `Failed` on mono, not
sure why exactly though.
Microsoft.Build.UnitTests.BackEnd.BuildManager_Tests.MultiProcReentrantProjectWithCallTargetDoesNotFail

.. since that doesn't work on mono yet.
.. from `mono/targets/$(MSBuildProjectName).Mono.targets`.
From the old `Mono.Override.targets` to
`mono/targets/MSBuild.Mono.targets`.
radical added 4 commits April 20, 2018 21:30
- built via the new mono/targets/MSBuild.Bootstrap.Mono.targets
- Update the project file to the new assembly paths
- Build initiated from mono/targets/MSBuild.Bootstrap.Mono.targets
- Update the project file to track new changes
- facade/tests/build.csproj -> build.proj: Manual build with Csc task
	- avoids some issues with nested csproj builds, and quicker

- TestTask: test for the location of assembly that got loaded

- built from mono/targets/MSBuild.Bootstrap.Mono.targets
- install.proj: this is roughly based on the earlier
  install-mono-prefix.sh, but many things changed with the new build
  system.

  `Usage: $ msbuild mono/build/install.proj /p:MonoInstallPrefix=<mono_install_prefix /p:Configuration=<Debug-MONO|Release-MONO>`
@radical
Copy link
Member Author

radical commented Apr 21, 2018

For anyone trying out this branch:

build: make all-mono
test: make test-mono
clean: make clean

install: ./install-mono-prefix.sh <mono_prefix>

radical added 2 commits April 21, 2018 04:10
.. bin dir.
We keep two canonical lists:

mono/build/all_files.canon.txt:
	- all the files that install.proj copies to $(MonoInstallPrefix)

 mono/build/remaining_files.canon.txt:
 	- files that were *not* copied from the bin dir in artifacts
@radical radical requested a review from marek-safar April 21, 2018 08:51
@marek-safar
Copy link
Member

@radical could you fill the description with more details and reasoning for this change

@radical
Copy link
Member Author

radical commented Apr 21, 2018

Updated the description.

@directhex
Copy link

It mooooooooostly achieves the objective. Builds if I uninstall cmake, on Ubuntu 18.04. dotnet/Nerdbank.GitVersioning#143

@radical
Copy link
Member Author

radical commented Apr 22, 2018

Added support for using a bootstrap msbuild.

@radical
Copy link
Member Author

radical commented Apr 22, 2018

@directhex maybe I can try to work around that. I tried just disabling the package reference, but that didn't work. Can you share the binary logs (artifacts/Release-MONO/log/Build*.binlog)? I want to see the trace and where in the build it was invoked.

@radical
Copy link
Member Author

radical commented Apr 22, 2018

hm maybe not.

radical added 7 commits April 22, 2018 20:13
.. default is to look in msbuildBinDir/Roslyn/ .

Essentially reversing:

    commit 157a0f4

    [mono] Roslyn's Csc task only looks in msbuild's directory for

    .. `Roslyn/csc.exe`. Earlier, it would fallback to looking for it in
    framework path which would lead it to the correct path for Mono.

    So, we need to hardcode the path now.
The previous commit which removed the `$(CscToolPath)` override, has the
side effect that the recent roslyn versions (2.7.x?) will use the shared
compiler, which doesn't yet work well on mono.

Earlier, `$(UseSharedCompilation)` was defaulting to true but it didn't
matter because roslyn didn't support the compiler server anyway.

Now it seems to work on netcore, but not (well enough) on mono. So, it
uses the compiler server if (essentially):

	1. `$(UseSharedCompilation)` is true (default)
	2. `$(CscToolPath)` is not overridden.

Since, the previous patch makes (2) true, we end up with the compiler
server being used. So, explicitly disable that for now.
This uses the msbuild in $PATH to build this bootstrap. This is mainly
to ensure that we get the corresponding Roslyn binaries too.

To create a bootstrap from the current build, install that, add to $PATH
and run the script.
- The canon lists are from a `-skipTests` build and would fail with a
  test run. So, we don't fail the install because of differences by
  default, which would apply to a local build where the user can see the
  issues.

- On CI let the differences fail the build so that they can be caught
- This had some extra files which were present in the my working
  directory but (correctly) weren't added to git.
@radical
Copy link
Member Author

radical commented Apr 23, 2018

After lots of rounds trying build-pkg+fix, I finally have a package that works from mono/mono#8388 . So, this PR is also ready for review+merge now.

There are other things to come right after this:

  • another upstream merge
  • SDK+resolver updates
  • 2018-02 port+updates

@radical radical merged commit 4ffef71 into mono:xplat-master Apr 23, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants