Support for Ruby 3.2.0 in prebuilt binaries

[frederikspang]

What version of protobuf and what language are you using?
Version: main
Language: Ruby

What operating system (Linux, Windows, ...) and version?
All

What runtime / compiler are you using (e.g., python version or gcc version)
Any

What did you do?
Install latest rubygem.

What did you expect to see
Prebuilt binary for Ruby 3.2.0 to be installed

What did you see instead?
It is installing with native extensions (ie. compiling)

Anything else we should know about your project / environment
Reopening #11407 with proper template, and because the issue has been closed.

Virtually, the rubygem needs to explicitly support and build for Ruby 3.2.0 as well as the others.
Currently the native prebuilt gems are build for >= 2.5, < 3.2.DEV
(see https://rubygems.org/gems/google-protobuf/versions/3.21.12-x86_64-linux for instance)

We should add support for 3.2.0 as it has been released in December.

[deannagarcia]

As stated in the bug you linked to, the workaround for this is to use the ruby source gem instead of the compiled binary. We are working on 22.0 currently, which will only release ruby source gems making them default compatible with ruby 3.2.

Unfortunately I don't think we will be updating the ruby gem binary for 21.x given the impending release.

[pond]

@deannagarcia and how long until that is released?

Bundler does not like keeping ruby under PLATFORMS these days and tends to want to force in specific architectures; and some of the components in our gem stack, including grpc, have extremely long build times when built from source - which costs us a fortune in CI time. Compilation can also be fragile. So - we don't want to keep the hack we have in our CI Dockerfile forcing Bundler into a ruby platform any longer than we must.

[deannagarcia]

We are currently aiming to release next week.

When we release only source gems will that solve your problem or will you still have the same issue?

[pond]

@deannagarcia Oh cool, that's not long to wait at all, thanks.

Source-only gems should be OK for protobuf since it builds fairly quickly (GRPC is around 10 minutes or more!). There's always a risk of failure when building native components but it sounds like you don't want to have to manage a wide range of pre-built binaries or have to keep the builds on a Ruby treadmill and that definitely makes sense.

[mrmason]

We're also in the position of wanting this - building from source takes way to long on CI/CD - plus much more bloated images with the build tools. Looking forward to an officially supported gem - we're not going to go production without it and really want to test YJIT. Thanks for all the hard work everyone involved.

[frederikspang]

While source-only gems will mitigate the problem, it is in my opinion a suboptimal solution.
I do not have build-times for protobuf, but there IS some overhead, and as @mrmason mentions (As I am writing this), the low-resource CI/CD pipelines, will produce a overhead both in image-bloating as well as build times for us as well.

I will strongly suggest, that we keep up the support for prebuilt gems. If not with the existing CI pipeline, then maybe we could rethink the build-setup a bit, to make it easier in the future, to support future Ruby versions (And maybe architectures etc.)?

[johnnyshields]

PR raised for supporting fat binaries on Ruby 3.2: #11592

[mockdeep]

From what I can tell the build time for protobuf is negligible. It seems like precompiled binaries add extra maintenance overhead and delay our ability to upgrade Ruby, so I'm totally in favor of removing them. It's easy to cache things on CI, and there are loads of other gems that take much longer to compile.

[pond]

I note that it was estimated there'd be 3.22 within the week, but it's been more than two weeks and there is still no 3.22 other than a release candidate from Feb-2, which other Google gems do not select via their gemspec, so we are still unable to use this in CI.

Is there an update, please, on when this will be done? It's kinda dragging and the maintenance burden and extra CI time required by being forced into a ruby platform is really expensive.

[deannagarcia]

Why aren't you able to use 3.22-rc2?

Unfortunately our release process often takes multiple release candidate iterations and we are actually trying to get support for binary gems into 3.22. We are aiming to get the full release out next week, but if we see issues with release candidates it could get pushed back.

[pond]

We don't use protobuf directly. It's pulled in as a dependency of GRPC, which uses ~> 3.21 AFAICT and that won't select RCs.

[johnnyshields]

@pond you can add protobuf 3.22-rc2 to your Gemfile for now.

[pond]

@johnnyshields I appreciate the suggestion, but then I have to add a task to my backlog to remove it again, and/or keep watching Google Protobuf every day, just so I can remove the hack.

We don't hack our software and we don't put alpha, beta or release candidate gems into Production.

[pond]

@deannagarcia A little ironic that after all this it's going to be a binary release anyway 😂 but thanks for the update and:

We are aiming to get the full release out next week, but if we see issues with release candidates it could get pushed back.

Understood.

[johnnyshields]

We don't hack our software and we don't put alpha, beta or release candidate gems into Production.

Then you're doing it wrong 😆

[pond]

I see 3.22.0 has now been released - thanks; we've updated to that and the CI builds are running again. Yay!