provide glibc as another BinaryTool #5662
Comments
|
Note to self that this is how clang finds those directories -- |
cosmicexplorer
added a commit
that referenced
this issue
Jul 2, 2018
… native backend subsystem tests (#5943) ### Problem The native backend subsystems tests introduced in #5490 are still skipped, complaining about `crti.o` on linux, which is part of libc. See #5662 -- `clang`'s driver will find the directory containing that file on travis, but `gcc` won't. We should make a way to find this file (which is necessary for creating executables) so we can unskip the native backend testing. ### Solution - Fix a mistake in #5780 -- we did not check the correct directory with `os.path.isdir()`, so we never found the `LLVM` BinaryTool when downloading it from the LLVM download page. - Find libc using the new `LibcDev` subsystem. This uses the option `--libc-dir`, if provided, or finds an installation of libc with `crti.o` by invoking `--host-compiler` on the host once to get its search directories (this is necessary on travis, due to ubuntu's nonstandard installation location). - Expand the definition of executables, compilers, and linkers in `src/python/pants/backend/native/config/environment.py` to cover everything needed to actually compile, and give them the ability to generate an environment suitable for passing into `subprocess.Popen()`. - Introduce `GCCCCompiler`, `GCCCppCompiler`, `LLVMCCompiler`, and `LLVMCppCompiler` to differentiate between the two different compilers we have available for each language. - Expose the libc lib directory to the compilers we create in `native_toolchain.py`. - Unskip tests in `test_native_toolchain.py` from #5490. - Make the default `CCompiler` and `CppCompiler` into clang, for no particular reason (it will pass CI with gcc as well). The different compilers we can use will likely be denoted using variants in the future, but this solution allows us to separate the resources generated from each subsystem (`GCC`, `LLVM`, `Binutils`, `XCodeCLITools`) from a fully-functioning compiler that can be invoked (because actually invoking either clang or gcc requires some resources from the other, e.g. headers and libraries). Now, each binary tool only does the job of wrapping the resources it contains, while `native_toolchain.py` does the job of creating either a clang or a gcc compiler that we can invoke on the current host (with all necessary headers, libs, etc). ### Result The native toolchain tests from #5490 can be unskipped, and we can invoke our toolchain on almost any linux installation without further setup. The tests in `test_native_toolchain.py` now pass in CI, and as we iterate on the native backend, we will continue to have end-to-end testing for both of our compilers, on osx as well, regardless of whichever one we choose to use for `python_dist()` compilation.
|
Resolved with #5943. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
This should have been done earlier, but glibc headers were available on all Linux systems tested in standard-enough places for clang to find. However, gcc does have issues finding these headers in Travis (see this FIXME from #5490) -- while that FIXME says to use clang's driver to find include directories for either clang or gcc, we should probably instead just provide the entire toolchain.
Reasoning: Trying to invoke
gccwill fail on travis, butclangwill work, because the clang we package is able to find object files and headers that gcc cannot: see this CI failure, I was able to "fix" that by debugging on anubuntu:trustyimage, but found other errors: specifically, clang's preprocessor could find/usr/include/x86_64-linux-gnu, while gcc was hopelessly stumbling around for/usr/include/x86_64-pc-linux-gnu.This would need #5661 to affect
python_dist()targets (otherwise it is difficult to perform this sort of configuration).The text was updated successfully, but these errors were encountered: