Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix static linking when system libraries installed #93

Merged
merged 2 commits into from
Sep 9, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,11 @@ on:
jobs:
compile-and-test-system-dependencies:
needs: ["build-cruby-gem"]
name: System Dependencies - Ruby ${{ matrix.ruby }} - libre2 ABI version ${{ matrix.libre2.soname }}
name: System Dependencies - ${{ matrix.sys }} vendored libs - Ruby ${{ matrix.ruby }} - libre2 ABI version ${{ matrix.libre2.soname }}
runs-on: ubuntu-20.04
strategy:
matrix:
sys: ["enable", "disable"]
ruby:
- '3.2'
- '3.1'
Expand Down Expand Up @@ -62,7 +63,9 @@ jobs:
with:
name: cruby-gem
path: gems
- run: ./scripts/test-gem-install gems --enable-system-libraries
- name: Add a symlink for libre2
run: ln -s /usr/lib/libre2.so `ruby -e "puts RbConfig::CONFIG['exec_prefix']"`/lib/libre2.so
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is exec_prefix on these runners?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't looked, but according to https://github.com/mudge/re2/actions/runs/6126602337/job/16630989217?pr=93, I suspect it's something like /opt/hostedtoolcache/Ruby/3.1.4/x64. ruby/setup-ruby#140 describes another way of retrieving the prefix.

- run: ./scripts/test-gem-install gems --${{matrix.sys}}-system-libraries

compile-and-test-vendored-dependencies:
name: Vendored Dependencies - Ruby ${{ matrix.ruby }} - ${{ matrix.runs-on }}
Expand Down
70 changes: 61 additions & 9 deletions ext/re2/extconf.rb
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,7 @@ def with_temp_dir
RbConfig::CONFIG["CXX"] = ENV["CXX"]
end

def build_extension
def build_extension(static_p = false)
$CFLAGS << " -Wall -Wextra -funroll-loops"

# Pass -x c++ to force gcc to compile the test program
Expand All @@ -128,7 +128,7 @@ def build_extension
have_header("stdint.h")
have_func("rb_str_sublen")

unless have_library("re2")
if !static_p and !have_library("re2")
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we basically using our own variant of have_library when using MiniPortile2 here because we need to override the linker config to give our vendored dependencies priority?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we're linking the .a explicitly instead of using -lre2. The latter will pick up the system dependency if it's installed in a certain location, and I don't believe there's anything we can do about that.

abort "You must have re2 installed and specified with --with-re2-dir, please see https://github.com/google/re2/wiki/Install"
end

Expand Down Expand Up @@ -326,14 +326,65 @@ def build_with_system_libraries
-labsl_time_zone
].freeze

def add_static_ldflags(flags)
static_flags = flags.split
def libflag_to_filename(ldflag)
case ldflag
when /\A-l(.+)/
"lib#{Regexp.last_match(1)}.#{$LIBEXT}"
end
end

# This method does a number of things to ensure the final shared library
# is compiled statically with the vendored libraries:
#
# 1. For -L<path> flags, ensure that any `ports` paths are prioritized just
# in case there are installed libraries that might take precedence.
# 2. For -l<lib> flags, convert the library to the static library with a
# full path and substitute the absolute static library. For example,
# -lre2 maps to /path/to/ports/<arch>/libre2/<version>/lib/libre2.a.
#
# This is needed because when building the extension, Ruby appears to
# insert `-L#{RbConfig::CONFIG['exec_prefix']}/lib` first. If libre2 is
# in installed in that location then the extension will link against the
# system library instead of the vendored library.
def add_flag(arg, lib_paths)
case arg
when /\A-L(.+)\z/
# Prioritize ports' directories
lib_dir = Regexp.last_match(1)
$LIBPATH =
if lib_dir.start_with?(PACKAGE_ROOT_DIR + "/")
[lib_dir] | $LIBPATH
else
$LIBPATH | [lib_dir]
end
when /\A-l./
filename = libflag_to_filename(arg)

added = false
lib_paths.each do |path|
static_lib = File.join(path, filename)

next unless File.exist?(static_lib)

$LDFLAGS << " " << static_lib
added = true
break
end

append_ldflags(arg.shellescape) unless added
else
append_ldflags(arg.shellescape)
end
end

def add_static_ldflags(flags, lib_paths)
static_flags = flags.strip.shellsplit

if MiniPortile.windows?
static_flags.each { |flag| append_ldflags(flag) unless ABSL_LDFLAGS.include?(flag) }
ABSL_LDFLAGS.each { |flag| append_ldflags(flag) }
static_flags.each { |flag| add_flag(flag, lib_paths) unless ABSL_LDFLAGS.include?(flag) }
ABSL_LDFLAGS.each { |flag| add_flag(flag, lib_paths) }
else
static_flags.each { |flag| append_ldflags(flag) }
static_flags.each { |flag| add_flag(flag, lib_paths) }
end
end

Expand Down Expand Up @@ -371,8 +422,9 @@ def build_with_vendored_libraries

raise 'Unable to run pkg-config --libs --static' unless $?.success?

add_static_ldflags(flags)
build_extension
lib_paths = [File.join(abseil_recipe.path, 'lib'), File.join(re2_recipe.path, 'lib')]
add_static_ldflags(flags, lib_paths)
build_extension(true)
end

if config_system_libraries?
Expand Down