FreeBSD support

[asomers]

Describe the enhancement
Support building the runner on FreeBSD

Additional information
I think FreeBSD has all the libraries that the runner needs. And while the dotnet-sdk isn't availble from Microsoft, there is already a package for it: lang/linux-dotnet-sdk . So it should be possible to build the runner on FreeBSD.

[TingluoHuang]

We can't really do that for mainly 2 reasons.

• Supportability, without office support from dotnet core team, debugging any native dependency problem will be a nightmare.
• Legal, we (GitHub) might run into legal problem when we consume and redistribute 3rd party open source project.

But, we are totally fine with forking this repository and try yourselves, we are generally doing do on holding server/client compat.

Hope this can help. 😄

[PrivatePuffin]

Addition:
lang/linux-dotnet-sdk is based on a very old version. Considering the DotNet Core team is working on version 5 by now, I doubt anyone is willing to spend time debuging it, as @TingluoHuang explained.

[TingluoHuang]

We can add support if we do #243 or Dotnet 5 supports FreeBSD

[Neilpang]

Sorry to bother you.
but I just created an action to make it possible to use FreeBSD in your workflow.

https://github.com/vmactions/freebsd-vm

[davidchisnall]

I'd love to see FreeBSD support, but there's a long tail of other platforms. For Microsoft/snmalloc, we support Windows, macOS, Linux, OpenEnclave, and FreeBSD and have had external contributions for NetBSD, Haiku, OpenBSD, DragonflyBSD and Solaris. We support x86, ARM, MIPS, and RISC-V, (most in both 32- and 64-bit variants) and have CHERI versions in development for ARM, MIPS and RISC-V.

We would love to spin up CI for more of these, and can run a lot of them in Azure to provide self-hosted runners (at least for a weekly test run, if not on every PR), but porting .NET to all of them and having supported releases is a phenomenal amount of work, particularly over the next couple of years as we grow the amount of CI that we want to run on experimental hardware.

It would probably be more valuable to the community to have the protocol that the runners need to speak documented so that third-party contributors can implement something for operating systems that don't have first-party support. The only thing that I need a runner to do for our CI to work is install some packages, download the code, and run a shell script, then report the results. A minimal C++ implementation that I can compile to a statically-linked binary to provide this minimal environment would be ideal for our purposes.

The fact that the only way of getting FreeBSD / OpenBSD / Solaris CI on GitHub actions currently is to spin up a Mac worker and run the desired OS in a VM is quite sad, especially since the initial launch announcement said that other operating systems would be supported.

[ChristopherHX]

You might be interested in a github action compatible runner (at least github.com works) which runs on freebsd and other systems without vm. It was a bit tricky to replicate the protocol, but finally it connected to the github action service.
You can use

• run steps
• composite run steps
• node actions if you have node in your PATH prior launching

[davidchisnall]

@ChristopherHX, thank you, this is fantastic! I can confirm that your runner works really well on FreeBSD. I have not yet done so, but it looks as if the --once flag should make it trivial to set up a jail that uses a ZFS clone that is discarded at the end of the run.

For anyone else looking to use GitHub actions with FreeBSD, I can thoroughly recommend @ChristopherHX's runner. It would be great if GitHub would provide a free runner pool using this for FreeBSD in addition to the Linux / Windows / macOS ones so that folks aren't using the macOS pool and vm-actions to run FreeBSD in VirtualBox on macOS and can just use it directly on a (much cheaper) Azure VM.

Any GitHub people reading this, feel free to ping me on Teams if you want help setting this up.

[davidchisnall]

I've written some scripts for running @ChristopherHX's version on FreeBSD. They use pot to create a jail backed by a ZFS dataset with the runner and any dependencies you want to install and then invoke this in a loop, with pot rolling back the ZFS dataset to the most recent snapshot in between each run. Scripts run with root privilege inside the jail and can do pretty much anything short of loading kernel modules (jails have a shared kernel). I've got it working on one of our repos with a VM providing runners in 12.2 and 13.0 FreeBSD jails. This may be useful for anyone else wanting self-hosted runners on FreeBSD. The scripts are very rough at the moment.

[michael-o]

I guess even though most issues/requirements were resolved, this is still not going forward?

[michael-o]

nvm

What's that?

[ykla]

I hope to support FreeBSD.

[Thefrank]

This sounds like a bit of a "chicken or egg" problem. Support for FreeBSD is not going to get added because there is not enough people asking for it so people won't add/use/want FreeBSD runners because of that.(*)

There might be an org size issue too. If your org is large enough to look around and go "FreeBSD would work great for THING-WE-DO because FIT/LICENSING/STAFF/ETC" your org already has the staff to support FreeBSD without outside help or will hire to fill gaps. FreeBSD's usage is, sadly, still rather niche (e.g., Netflix and NetApp use it mostly for getting the most out of KTLS, Sony uses it as a base for their PlayStation console, Apple originally used it for...)

From the above, as a financial standpoint for Microsoft, FreeBSD support would be the digital equivalent of a giant money pit. But with the pit, you can see you got a pit; with this all you see is red on your books.

*A current similar example for those that enjoy long reads is FreeBSD in dotNET.

[cederom]

+1 to add FreeBSD Actions / CI / Runner support. We are censored out from lots of projects where code / test / build / packaging could take place! Please add FreeBSD support! :-)

[bryanmacfarlane]

One solution is to port the runner to go. We already use it in many of our services in github/actions. We (actions engineering) have discussed and it's something on our radar.

[michael-o]

This sounds like a bit of a "chicken or egg" problem. Support for FreeBSD is not going to get added because there is not enough people asking for it so people won't add/use/want FreeBSD runners because of that.(*)

There might be an org size issue too. If your org is large enough to look around and go "FreeBSD would work great for THING-WE-DO because FIT/LICENSING/STAFF/ETC" your org already has the staff to support FreeBSD without outside help or will hire to fill gaps. FreeBSD's usage is, sadly, still rather niche (e.g., Netflix and NetApp use it mostly for getting the most out of KTLS, Sony uses it as a base for their PlayStation console, Apple originally used it for...)

From the above, as a financial standpoint for Microsoft, FreeBSD support would be the digital equivalent of a giant money pit. But with the pit, you can see you got a pit; with this all you see is red on your books.

*A current similar example for those that enjoy long reads is FreeBSD in dotNET.

That would also mean that Microsoft should stop using open source immediately because someone else paid (money or time) to develop it. Give and take.

[michael-o]

One solution is to port the runner to go. We already use it in many of our services in github/actions. We (actions engineering) have discussed and it's something on our radar.

That would be ideal, I never understood why C# was chosen.

[davidchisnall]

One solution is to port the runner to go

Good news everyone!

(Seriously, if you do decide on Go, please don't duplicate effort)

[cryptocode]

One solution is to port the runner to go. We already use it in many of our services in github/actions. We (actions engineering) have discussed and it's something on our radar.

Any updates on this? The project David mentioned seem to confirm its feasibility. Just say dotnot to dotnet for infra projects.

Users of GitHub (including Microsoft as mentioned earlier) currently have to resort to ugly kludges. As a result, countless projects are holding back on offering FreeBSD support when they could easily do so if only GH offered an easy runner solution.

[flavorjones]

Just dropping a note here for anyone who's interested, I've been using https://github.com/vmactions/freebsd-vm for the last few years and it's been a really solid option. It's slower than a native bsd runner would be, but it's sufficient for coverage for my use case.

[cederom]

Maybe in 2025 this can happen??

[bsdimp]

For what it's worth, Netflix uses FreeBSD for it's video streaming boxes. We recently migrated to github and are using all kinds of ugly things to do post-commit CI. Having native support for FreeBSD would be a game changer for us.

[cederom]

Exactly! So far we are still excluded from all sorts of automations like PR verification , build and runtime checks, release packages and images generation, tools and cross-platform verification, etc etc..

[silenius]

The lack of FreeBSD support is also problematic with the latest version of TailwindCSS, see tailwindlabs/tailwindcss#15731

[marten-seemann]

We've run into numerous FreeBSD problem in quic-go over the years, which could've easily caught if there was a way to run on FreeBSD in CI.

[cederom]

FreeBSD related CI / PR build test automation is missing in Apache NuttX RTOS (https://github.com/apache/nuttx). We have Linux, macOS, and Windows builds in standard tests. We recently reorganized CI builds into stages because of CI quota hits caused mainly because macOS is 10x expensive than Linux, optimization is in place for all platforms, build for ~1000 targets is done mainly with Linux, still even basic tests are not possible for FreeBSD.

Working with all sorts of Python tools in VirtualEnv is pain because we need to build all packages on-target as no packages are available due CI missing for FreeBSD. This takes time and causes lots of upstream problems because build cannot be verified prior release. With CI available packages could be verified and generated for FreeBSD and installed quickly as on other platforms.

[dbohdan]

@bsdimp If you're able to share, I am curious about Netflix's FreeBSD CI setup. Are you using marketplace/actions/cross-platform-action or a similar solution?

I would also like to see native FreeBSD CI on GitHub, but I want to bring attention to Cross-Platform GitHub Action as something you can use today. It has made FreeBSD, NetBSD, and OpenBSD CI viable for me and even the default for my Go projects. One caveat I can give is that I haven't used it for large projects with heavy resource usage in CI. I am not affiliated with the action, just a happy user.

[Thefrank]

GH Runners would be good for:

• Anyone looking to make/test FreeBSD NuGets/Binaries using .NET. FreeBSD can run dotNET. My workaround for dotNET items is using AZP on FreeBSD and either cloud or local hosting the agent depending on need.
  • This is more complicated as it would require Microsoft to provide NuGet feeds for FreeBSD. FreeBSD is community supported platform and not officially supported.
• Anyone looking to make/test FreeBSD Node projects. FreeBSD supports Node.
• Anyone looking to make/test FreeBSD Python projects. FreeBSD supports Python.
• Anyone looking to make/test FreeBSD Rust projects. FreeBSD supports Rust.
• Me wanting to port things to FreeBSD using GH if the project is hosted on GH.
• Other Microsoft projects: Onboard FreeBSD GitHub Action microsoft/msquic#3033
• ... etc

The current workaround is hacky, slow, and EXPENSIVE: Using OSX+byhve->FreeBSD.

F/OSS projects don't really make money and only having the workaround deters them from adopting FreeBSD.

I understand this is likely a resourcing issue but it still feels bad.

[emaste]

As many have highlighted, native FreeBSD support for GitHub Actions would greatly benefit a wide range of developers and teams. The FreeBSD Foundation recognizes the significance of this effort and is willing to contribute developer resources to help make it happen.