A bunch of improvements to native library builds #2049

alexrp · 2024-04-12T23:35:05Z

Adds OpenAL Soft native build script.
Extends platform coverage for most native packages.
Adds symbol stripping to most Linux/macOS native binaries.
Changes all native binaries to be built with -O3.
Builds SDL with dynamically-loaded X11 and Wayland support.
Excludes some unnecessary churn during native builds (tests, examples, etc).
Cleans up some of the build scripts to be table-driven.
Drops linux-x86 support (was never officially supported by .NET).

Contributes to #727.
Closes #2015.
Closes #2016.

Perksey · 2024-04-14T04:15:30Z

Just going to kick off the builds for the purposes of review - just close the PRs as they're opened, I'm only interested in the checkmarks.

Perksey

Nothing like a nice glass of Bailey's to go along with an evening code review (it's a good thing I found no comments because I drank the last bit and import prices in Texas are ridiculous)

@Beyley PTAL

build/nuke/Native/OpenALSoft.cs

build/nuke/Native/SDL2.cs

src/Native/Silk.NET.Assimp.Native/Silk.NET.Assimp.Native.csproj

These will mainly be used in CI workflows, so they're specific to Ubuntu 18.04 and 22.04.

Also clean up the build script.

Perksey

Latest changes are fine provided @Beyley agrees

The sndio backend takes a hard dependency on libsndio.so, unlike most other backends which load libraries dynamically. We don't want our libopenal.so to require users to install sndio.

This allows the build to pick them up and enable support.

This package is not multiarch-compatible in 22.04, so prioritize amd64.

…Fri Mar 15 00:10:50 PDT 2024; root:xnu-10063.101.17~1/RELEASE_ARM64_VMAPPLE

…P Sat Mar 9 04:50:38 UTC 2024

…ntu SMP Sat Mar 9 04:50:38 UTC 2024

Update OpenALSoft binaries

Update SDL2 binaries

Beyley

only thing left is just a missing comment, add that and ale li pona tawa mi, really nice work!

.github/workflows/assimp.yml

.github/workflows/glfw.yml

.github/workflows/openal-soft.yml

.github/workflows/sdl2.yml

Perksey · 2024-04-18T00:27:55Z

Maybe it's worth factoring out the Ubuntu extras as well into a bash script?

alexrp · 2024-04-18T00:57:08Z

Maybe it's worth factoring out the Ubuntu extras as well into a bash script?

The proper way to handle the duplication of logic in the native workflows is to switch them over to GitHub's reusable workflows feature, but that's a project for another day.

Beyley

pona

Perksey · 2024-04-18T02:51:09Z

Force merging because CLA is sleeping

…ader.Native. This was missed in #2049.

* Native: Also include *.json files in PRs (e.g. vk_swiftshader_icd.json). * Native/SwiftShader: Update submodule to 764410d. * Native/SwiftShader: Build for win-arm64. * Native/SwiftShader: Update packaging paths in Silk.NET.Vulkan.SwiftShader.Native. This was missed in #2049. * Native/Vulkan Loader: Build for win-arm64. Vulkan Loader has GAS-style assembly that is essential to it functioning correctly, so to make things simple, we build on Linux using llvm-mingw. * Native/SwiftShader: Expand on a comment in the NUKE build script. * Native: Use absolute tool paths in aarch64-w64-mingw32.cmake. * Native: Re-run CMake-based workflows when build/cmake/* changes are pushed. * Native/Vulkan Loader: Remove llvm-mingw archive after extraction. * Native/Vulkan Loader: Pass extra glob path to the win-arm64 binary. This is needed since the default globs only include *.so on Linux. * New binaries for SwiftShader on Darwin 23.4.0 Darwin Kernel Version 23.4.0: Fri Mar 15 00:10:50 PDT 2024; root:xnu-10063.101.17~1/RELEASE_ARM64_VMAPPLE * Native/Vulkan Loader: Update submodule to v1.3.281. This brings it in line with our Vulkan-Headers submodule. * Native/Vulkan-Loader: Update submodule to 6d9e995. This brings in a number of build system fixes that we need for win-arm64. This commit is just prior to the update to Vulkan v1.3.282. * Native/Vulkan Loader: Fix path to vulkan-1.dll for win-arm64. * New binaries for SwiftShader on Microsoft Windows 10.0.20348 * Native/Vulkan Loader: Fix path to vulkan-1.dll for win-arm64, take 2. * New binaries for Vulkan Loader on Darwin 23.4.0 Darwin Kernel Version 23.4.0: Fri Mar 15 00:10:50 PDT 2024; root:xnu-10063.101.17~1/RELEASE_ARM64_VMAPPLE * New binaries for Vulkan Loader on Linux 6.5.0-1018-azure #19~22.04.2-Ubuntu SMP Thu Mar 21 16:45:46 UTC 2024 * New binaries for Vulkan Loader on Microsoft Windows 10.0.20348 * New binaries for SwiftShader on Linux 6.5.0-1018-azure #19~22.04.2-Ubuntu SMP Thu Mar 21 16:45:46 UTC 2024 * Native: Simplify `git add` glob for changed native binaries. * Native: Simplify NuGet packaging projects. * Native: Set CMAKE_ASM_COMPILER in all toolchain files. --------- Co-authored-by: The Silk.NET Automaton <9011267+dotnet-bot@users.noreply.github.com>

alexrp linked an issue Apr 13, 2024 that may be closed by this pull request

.NET 6 arm64 builds #727

Closed

38 tasks

alexrp force-pushed the feature/natives branch 15 times, most recently from a427837 to fc5dacb Compare April 14, 2024 03:57

alexrp marked this pull request as ready for review April 14, 2024 04:00

alexrp requested a review from a team as a code owner April 14, 2024 04:00

Perksey approved these changes Apr 14, 2024

View reviewed changes

build/nuke/Native/OpenALSoft.cs Show resolved Hide resolved

build/nuke/Native/SDL2.cs Show resolved Hide resolved

build/nuke/Native/SDL2.cs Show resolved Hide resolved

src/Native/Silk.NET.Assimp.Native/Silk.NET.Assimp.Native.csproj Show resolved Hide resolved

Perksey requested a review from Beyley April 14, 2024 04:50

Perksey assigned Beyley Apr 14, 2024

alexrp force-pushed the feature/natives branch 2 times, most recently from 27c4f51 to e9d8ec6 Compare April 14, 2024 11:07

alexrp added 6 commits April 14, 2024 13:08

Native: Add CMake toolchain files for Linux cross-compilation.

2db4338

These will mainly be used in CI workflows, so they're specific to Ubuntu 18.04 and 22.04.

Native/Assimp: Build for linux-arm, linux-arm64, and win-arm64.

fadce2c

Also clean up the build script.

Native/Assimp: Don't build tests.

ced48cc

Native/Assimp: Strip Linux/macOS binaries after build.

dda5b48

Native/GLFW: Build for linux-arm and linux-arm64.

760df13

Also clean up the build script.

Native/GLFW: Don't build examples, tests, and docs.

59231e4

Perksey approved these changes Apr 14, 2024

View reviewed changes

alexrp and others added 15 commits April 14, 2024 19:06

Native/OpenAL Soft: Explicitly disable sndio support.

303e850

The sndio backend takes a hard dependency on libsndio.so, unlike most other backends which load libraries dynamically. We don't want our libopenal.so to require users to install sndio.

Native/OpenAL Soft: Ensure that D-Bus libraries are installed in CI.

fc18f2c

This allows the build to pick them up and enable support.

Native/OpenAL Soft: Fix library/PR name.

280c8e9

Native/OpenAL Soft: Only install JACK for amd64 in the workflow.

6835bb0

This package is not multiarch-compatible in 22.04, so prioritize amd64.

Native/SDL: Only install JACK for amd64 in the workflow.

68d6289

This package is not multiarch-compatible in 22.04, so prioritize amd64.

New binaries for SDL2 on Darwin 23.4.0 Darwin Kernel Version 23.4.0: …

4098bef

…Fri Mar 15 00:10:50 PDT 2024; root:xnu-10063.101.17~1/RELEASE_ARM64_VMAPPLE

New binaries for SDL2 on Microsoft Windows 10.0.20348

386a5df

Merge branch 'feature/natives' into ci/feature/natives/sdl2_bins

d6532ab

New binaries for OpenALSoft on Microsoft Windows 10.0.20348

a91fc8d

New binaries for SDL2 on Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SM…

1b7a212

…P Sat Mar 9 04:50:38 UTC 2024

Merge branch 'feature/natives' into ci/feature/natives/sdl2_bins

6689ce7

New binaries for OpenALSoft on Linux 6.5.0-1017-azure #17~22.04.1-Ubu…

ada5115

…ntu SMP Sat Mar 9 04:50:38 UTC 2024

Merge branch 'feature/natives' into ci/feature/natives/openalsoft_bins

9970dbf

Merge pull request #2114 from dotnet/ci/feature/natives/openalsoft_bins

aba6ecd

Update OpenALSoft binaries

Merge pull request #2113 from dotnet/ci/feature/natives/sdl2_bins

627d67a

Update SDL2 binaries

alexrp mentioned this pull request Apr 15, 2024

Build Vkd3d shader compiler native package for linux-arm and linux-arm64 #2126

Closed

Beyley requested changes Apr 18, 2024

View reviewed changes

.github/workflows/assimp.yml Show resolved Hide resolved

.github/workflows/glfw.yml Show resolved Hide resolved

.github/workflows/openal-soft.yml Show resolved Hide resolved

.github/workflows/sdl2.yml Show resolved Hide resolved

Native/Workflows: Add comments explaining APT sources.

a6252a0

Beyley approved these changes Apr 18, 2024

View reviewed changes

Perksey enabled auto-merge (squash) April 18, 2024 02:51

Perksey disabled auto-merge April 18, 2024 02:51

Perksey merged commit aea6c51 into main Apr 18, 2024
8 of 27 checks passed

alexrp added a commit that referenced this pull request Apr 18, 2024

Native/SwiftShader: Update packaging paths in Silk.NET.Vulkan.SwiftSh…

a750258

…ader.Native. This was missed in #2049.

Perksey deleted the feature/natives branch April 18, 2024 04:08

alexrp added a commit that referenced this pull request Apr 18, 2024

Native/SwiftShader: Update packaging paths in Silk.NET.Vulkan.SwiftSh…

af4eab3

…ader.Native. This was missed in #2049.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

A bunch of improvements to native library builds #2049

A bunch of improvements to native library builds #2049

alexrp commented Apr 12, 2024 •

edited

Loading

Perksey commented Apr 14, 2024

Perksey left a comment

Perksey left a comment

Beyley left a comment

Perksey commented Apr 18, 2024

alexrp commented Apr 18, 2024

Beyley left a comment

Perksey commented Apr 18, 2024

A bunch of improvements to native library builds #2049

A bunch of improvements to native library builds #2049

Conversation

alexrp commented Apr 12, 2024 • edited Loading

Perksey commented Apr 14, 2024

Perksey left a comment

Choose a reason for hiding this comment

Perksey left a comment

Choose a reason for hiding this comment

Beyley left a comment

Choose a reason for hiding this comment

Perksey commented Apr 18, 2024

alexrp commented Apr 18, 2024

Beyley left a comment

Choose a reason for hiding this comment

Perksey commented Apr 18, 2024

alexrp commented Apr 12, 2024 •

edited

Loading