Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add the default solib dir to the rpath for cc_imports with transitions #14272

Closed
wants to merge 2 commits into from
Closed

Add the default solib dir to the rpath for cc_imports with transitions #14272

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
fbc1ec3
Add the default solib dir to the rpath for cc_imports with transitions
quval Nov 14, 2021
b670747
Windows paths seem different - be less strict
quval Nov 14, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions src/main/java/com/google/devtools/build/lib/rules/cpp/LibrariesToLinkCollector.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -332,6 +332,14 @@ private void addDynamicInputLinkOptions(

rpathRootsForExplicitSoDeps.add(
rpathRoot + dotdots + libDir.relativeTo(commonParent).getPathString());

// Unless running locally, libraries will be available under the root relative path, so we
// should add that to the rpath as well.
if (inputArtifact.getRootRelativePathString().startsWith("_solib_")) {
PathFragment artifactPathUnderSolib = inputArtifact.getRootRelativePath().subFragment(1);
rpathRootsForExplicitSoDeps.add(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your patience.

I have a few questions. Running the test locally I get this:
Without your change:
Test 1:
-Wl,-rpath,$ORIGIN/../_solib_k8/_U_S_Sa_Cfoo___Ua
Test 2:
-Wl,-rpath,$ORIGIN/../_solib_k8/../../../k8-fastbuild-ST-1556fd9fdf05/bin/_solib_k8/_U_S_Sa_Cfoo___Ua

With your change:
Test 1:
-Wl,-rpath,$ORIGIN/../_solib_k8/_U_S_Sa_Cfoo___Ua
Test 2:

A) -Wl,-rpath,$ORIGIN/../_solib_k8/../../../k8-fastbuild-ST-1556fd9fdf05/bin/_solib_k8/_U_S_Sa_Cfoo___Ua
B) -Wl,-rpath,$ORIGIN/../_solib_k8/_U_S_Sa_Cfoo___Ua

IIUC if we are running locally we put the symlink under A) when there is a transition. Do we also put the symlink in B) every time, where in our codebase?

When we are not running locally A) won' t exist but B) will exist, where in the codebase do we put the symlink in B)?

Thanks!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for getting to this!

Your understanding is the same as mine: A is where the symlink exists locally with a transition, and B is where it exists otherwise (no transition or not running locally). As far as I remember, B doesn't exist when running locally.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you know why A) is not created with a transition running remotely?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

k8-fastbuild... (both with and without transitions) aren't created for remote execution or under the runfiles directory, and the correct versions of shared libraries are found under the solib directory instead.

rpathRoot + artifactPathUnderSolib.getParentDirectory().getPathString());
}
}

librarySearchDirectories.add(inputArtifact.getExecPath().getParentDirectory().getPathString());
Expand Down
124 changes: 124 additions & 0 deletions src/test/java/com/google/devtools/build/lib/rules/cpp/CcImportBaseConfiguredTargetTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@
import com.google.devtools.build.lib.analysis.util.BuildViewTestCase;
import com.google.devtools.build.lib.collect.nestedset.NestedSet;
import com.google.devtools.build.lib.packages.util.Crosstool.CcToolchainConfig;
import java.util.List;
import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
Expand Down Expand Up @@ -405,4 +406,127 @@ private void setupTestCcImportLoadedThroughMacro(boolean loadMacro) throws Excep
getAnalysisMock().ccSupport().getMacroLoadStatement(loadMacro, "cc_import"),
"cc_import(name='a', static_library='a.a')");
}

@Test
public void testCcImportWithSharedLibraryAddsRpathEntry() throws Exception {
AnalysisMock.get()
.ccSupport()
.setupCcToolchainConfig(
mockToolsConfig,
CcToolchainConfig.builder().withFeatures(CppRuleClasses.SUPPORTS_DYNAMIC_LINKER));
useConfiguration("--cpu=k8");
ConfiguredTarget target =
scratchConfiguredTarget(
"a",
"foo",
starlarkImplementationLoadStatement,
"cc_import(name = 'foo', shared_library = 'libfoo.so')");
scratch.file(
"bin/BUILD",
"cc_binary(name='bin', deps=['//a:foo'])");

Artifact dynamicLibrary =
target
.get(CcInfo.PROVIDER)
.getCcLinkingContext()
.getLibraries()
.getSingleton()
.getResolvedSymlinkDynamicLibrary();
Iterable<Artifact> dynamicLibrariesForRuntime =
target
.get(CcInfo.PROVIDER)
.getCcLinkingContext()
.getDynamicLibrariesForRuntime(/* linkingStatically= */ false);
assertThat(artifactsToStrings(ImmutableList.of(dynamicLibrary)))
.containsExactly("src a/libfoo.so");
assertThat(artifactsToStrings(dynamicLibrariesForRuntime))
.containsExactly("bin _solib_k8/_U_S_Sa_Cfoo___Ua/libfoo.so");

ConfiguredTarget main = getConfiguredTarget("//bin:bin");
Artifact mainBin = getBinArtifact("bin", main);
CppLinkAction action = (CppLinkAction) getGeneratingAction(mainBin);
List<String> linkArgv = action.getLinkCommandLine().arguments();
assertThat(linkArgv).contains("-Wl,-rpath,$ORIGIN/../_solib_k8/_U_S_Sa_Cfoo___Ua");
}

@Test
public void testCcImportWithSharedLibraryWithTransitionAddsRpathEntry() throws Exception {
AnalysisMock.get()
.ccSupport()
.setupCcToolchainConfig(
mockToolsConfig,
CcToolchainConfig.builder().withFeatures(CppRuleClasses.SUPPORTS_DYNAMIC_LINKER));
useConfiguration("--cpu=k8");
ConfiguredTarget target =
scratchConfiguredTarget(
"a",
"foo",
starlarkImplementationLoadStatement,
"cc_import(name = 'foo', shared_library = 'libfoo.so')");

scratch.file(
"bin/custom_transition.bzl",
"def _custom_transition_impl(settings, attr):",
" _ignore = settings, attr",
"",
" return {'//command_line_option:copt': ['-DFLAG']}",
"",
"custom_transition = transition(",
" implementation = _custom_transition_impl,",
" inputs = [],",
" outputs = ['//command_line_option:copt'],",
")",
"",
"def _apply_custom_transition_impl(ctx):",
" cc_infos = []",
" for dep in ctx.attr.deps:",
" cc_infos.append(dep[CcInfo])",
" merged_cc_info = cc_common.merge_cc_infos(cc_infos = cc_infos)",
" return merged_cc_info",
"",
"apply_custom_transition = rule(",
" implementation = _apply_custom_transition_impl,",
" attrs = {",
" '_whitelist_function_transition': attr.label(",
" default = '//tools/allowlists/function_transition_allowlist',",
" ),",
" 'deps': attr.label_list(cfg = custom_transition),",
" },",
")");
scratch.overwriteFile(
"tools/allowlists/function_transition_allowlist/BUILD",
"package_group(",
" name = 'function_transition_allowlist',",
" packages = ['//...'],",
")");
scratch.file(
"bin/BUILD",
"load(':custom_transition.bzl', 'apply_custom_transition')",
"cc_library(name='lib', deps=['//a:foo'])",
"apply_custom_transition(name='transitioned_lib', deps=[':lib'])",
"cc_binary(name='bin', deps=[':transitioned_lib'])");

Artifact dynamicLibrary =
target
.get(CcInfo.PROVIDER)
.getCcLinkingContext()
.getLibraries()
.getSingleton()
.getResolvedSymlinkDynamicLibrary();
Iterable<Artifact> dynamicLibrariesForRuntime =
target
.get(CcInfo.PROVIDER)
.getCcLinkingContext()
.getDynamicLibrariesForRuntime(/* linkingStatically= */ false);
assertThat(artifactsToStrings(ImmutableList.of(dynamicLibrary)))
.containsExactly("src a/libfoo.so");
assertThat(artifactsToStrings(dynamicLibrariesForRuntime))
.containsExactly("bin _solib_k8/_U_S_Sa_Cfoo___Ua/libfoo.so");

ConfiguredTarget main = getConfiguredTarget("//bin:bin");
Artifact mainBin = getBinArtifact("bin", main);
CppLinkAction action = (CppLinkAction) getGeneratingAction(mainBin);
List<String> linkArgv = action.getLinkCommandLine().arguments();
assertThat(linkArgv).contains("-Wl,-rpath,$ORIGIN/../_solib_k8/_U_S_Sa_Cfoo___Ua");
}
}