Mac support: Bazel plugin doesn't work with C++ std lib and repositories

[gruszczy]

I have checked and bazel binary is set correctly in the settings.

#include

Would not be detected properly by CLion. I have some repositories (both new_Http_repository and new_local_repository, for things like gtest, easylogging and llvm), neither is detected by CLion now.

Lots of warnings:

Warning:cc target //:type_inference_visitor_test does not depend on exactly 1 cc toolchain. Found 0 toolchains.
Warning:cc target //:parser_test does not depend on exactly 1 cc toolchain. Found 0 toolchains.
Warning:cc target @easylogging//:main does not depend on exactly 1 cc toolchain. Found 0 toolchains.
Warning:cc target //:runtime does not depend on exactly 1 cc toolchain. Found 0 toolchains.
Warning:cc target //:parser does not depend on exactly 1 cc toolchain. Found 0 toolchains.
Warning:cc target //:classes does not depend on exactly 1 cc toolchain. Found 0 toolchains.
Warning:cc target @llvm//:main does not depend on exactly 1 cc toolchain. Found 0 toolchains.
Warning:cc target //:shwifty does not depend on exactly 1 cc toolchain. Found 0 toolchains.
Warning:cc target //:common does not depend on exactly 1 cc toolchain. Found 0 toolchains.
Warning:cc target //:ast does not depend on exactly 1 cc toolchain. Found 0 toolchains.
Warning:cc target @gtest//:main does not depend on exactly 1 cc toolchain. Found 0 toolchains.
Warning:cc target //:lexer does not depend on exactly 1 cc toolchain. Found 0 toolchains.
Warning:cc target //:expression_type_visitor does not depend on exactly 1 cc toolchain. Found 0 toolchains.
Warning:cc target //:codegen_test does not depend on exactly 1 cc toolchain. Found 0 toolchains.
Warning:cc target //:type_inference_visitor does not depend on exactly 1 cc toolchain. Found 0 toolchains.
Warning:cc target //:lexer_test does not depend on exactly 1 cc toolchain. Found 0 toolchains.
Warning:cc target //:functions does not depend on exactly 1 cc toolchain. Found 0 toolchains.
Warning:cc target //:jit does not depend on exactly 1 cc toolchain. Found 0 toolchains.
Warning:cc target //:ast_compare_visitor does not depend on exactly 1 cc toolchain. Found 0 toolchains.
Warning:cc target //:type does not depend on exactly 1 cc toolchain. Found 0 toolchains.
Warning:cc target //:codegen does not depend on exactly 1 cc toolchain. Found 0 toolchains.

Unfortunately there is no explanation about what this means and how to fix this. I selected the default toolchain.

Symbols get properly detected (both std lib and my local repositories) when I use hand crafted cmake, which I use specifically for CLion.

[matthias-kraft]

+1

[danieldanciu]

+1

[jvoung]

Is there an example project we could try loading?
What platform? (OSX, Linux, WIndows)?
What do some of the "intellij-info.txt" files look like? For example, with tensorflow, after bazel sync I have "bazel-bin/tensorflow/core/session_options.intellij-info.txt" for cc_library. That cc_library has a deps on target "@local_config_cc//:cc-compiler-k8", which is a cc_toolchain.

[danieldanciu]

Thanks for helping out, Jan!
I'm running on a Mac laptop (Sierra 10.12.4), don't know about the others.

I attached my *intellij-info.txt file. The cc_toolchain deps look like this:

"@local_config_cc//:cc-compiler-darwin_x86_64"

I'll attempt to repro this with a toy project and will update the issue if successful.

vmu_eval_synthetic_main.intellij-info.txt

[gruszczy]

I'm on a Mac laptop too.

[danieldanciu]

I managed to repro this with a trivial HelloWorld project, which I uploaded here:

https://github.com/danieldanciu/clion-bazel

Attached is the bazel-bin/hello.intellij-info.txt file.
hello.intellij-info.txt

Screenshot also attached.

[jvoung]

Thanks for the repro! I think the issue is that the plugin needs to be able to get toolchain info from "apple_cc_toolchain" rule kinds and not just "cc_toolchain" rule kinds.

[danieldanciu]

Thanks for looking into this, Jan. Do you have a feeling on how much work it is to get toolchain info from apple_cc_toolchain? Are there any workarounds we could use to make bazel/clion work on our macs?

[brendandouglas]

Separately to the apple_cc_toolchain issue, a quick note that we currently only support Linux.

We've heard anecdotally that the Bazel plugin already basically works on Macs, but it's not something we've tested. We do intend to support it in the future, but it's not currently a top priority.

[siddharthab]

+1 for having tested CLion integration on Mac.

[siddharthab]

I tried my hand at this, but because I don't have visibility into the true repo, I can not send an actual PR against the current code.

The combination of these two commits was enough to get the CLion plugin working on Mac with the default toolchain.
https://github.com/siddharthab/bazel/commit/e39ef01505fc14a016c437e232b7fb9c14121f1a
https://github.com/siddharthab/intellij/commit/b58e45d256a836908abea92aea4f9fc53553d89a

I understand that the aspect files have moved from the bazel repo to be shipped inside CLion, but my purpose here is to test the change in principal.

Note that a much simpler workaround is to provide a cc_toolchain (not apple_cc_toolchain) using crosstool_top build flag in your project view file (.bazelproject).

With the default toolchain, there are still some kinks. The includes and auto completes work fine, but some classes are not found inside the std namespace. See the attached screenshot.

[matthias-kraft]

Hi Siddharthab,

thanks a lot for looking into this. I have little experience using crosstool files, would you mind providing an example of how you define your cc_toolchain and include it in .bazelproject in this instance?

[jgavris]

+1 @siddharthab are you planning on contributing those changes to bazel / intellij plugin?

[siddharthab]

@jgavris I think this repo is a read-only mirror of their true repo, so I am not sure how to contribute directly, but the change is really small and should not interfere with anything else. So if the team is interested, they will be able to fix it soon.

@mkraft89 I will not be able to say how the cc_toolchain is defined as someone else in my company did it, and there are many specific customizations in it, but you can try to follow the guide from bazel team on how to define your own toolchain.

As for the .bazelproject file, you can access it for an existing project using 'Bazel > Local Project View File'. For a new project, you will be presented the option to create/edit the file as part of the configuration. Here's a minimal example:

directories:
  src/

targets:
  //src/...:all

build_flags:
  --crosstool_top=//custom_toolchain

[siddharthab]

Update on the namespace std missing issue when using the default toolchain with the suggested patch.

Looks like LLVM needs either __clang__ or __GNUC__ defined to process the std namespace correctly (example). CMake based CLion projects define both of these macros, but Bazel plugin does not set either of these. And hence, the versioned std namespace gets mapped to the global namespace in LLVM.

If I define either of these macros manually at the top of my file, CLion is able to parse the file correctly. Here's the same example from before, now working perfectly for the std::namespace.

So now we know the exact things to get CLion working on Mac, at least at first glance.

Cheers!

[Timmmm]

I was going to post a message asking for help from @siddharthab (CROSSTOOL isn't exactly easy to follow) but amazingly I actually got this to work, at least as far as I can tell.

The approach I took was to make a hello-world project, and then copy the toolchain files out of its build directory. Something like this:

mkdir foo
cd foo
touch WORKSPACE
mkdir main
echo "int main() {}" >> main/main.cpp
echo "cc_binary( name = "foo", src = ["main.cpp] )" >> main/BUILD

bazel build //main/...:all
cd bazel-foo/external/local_config_cc
ls

There are lots of useful files in there. Now create your actual project with a structure like this:

├── custom_toolchain
│  ├── BUILD
│  ├── cc_wrapper.sh
│  ├── CROSSTOOL
│  ├── dummy_toolchain.bzl
│  ├── libtool
│  ├── make_hashed_objlist.py
│  ├── osx_archs.bzl
│  ├── WORKSPACE
│  ├── wrapped_ar
│  ├── wrapped_clang
│  ├── wrapped_clang_pp
│  ├── xcode-locator-bin
│  └── xcrunwrapper.sh
├── main
│  ├── BUILD
│  └── foo.cpp
└── WORKSPACE

The main folder and WORKSPACE are as before. All of the contents of custom_toolchain are copied from local_config_cc except BUILD and CROSSTOOL. BUILD I based on the one from local_config_cc but simplified it a bit (don't care about iOS etc.).

I couldn't get the CROSSTOOL from local_config_cc to work at all - it always gives errors about %sdk_dir not being defined or something. Instead I extracted the Apple bits from this test CROSSTOOL in the bazel repo.

I also had to change osx_cc_wrapper.sh to cc_wrapper.sh (no idea why) and I added

unfiltered_cxx_flag: "-D__clang__"

to avoid the std:: issue mentioned above.

I think those were the only changes I made. BUILD and CROSSTOOL files attached. It seems to work but I presume there is a reason they added apple_cc_toolchain (and then didn't update all the places that assumed cc_toolchain, ugh.) so I guess this may have broken some Apple-related things. Who knows.

[Timmmm]

Ugh you can't attach text files apparently? BUILD:

package(default_visibility = ["//visibility:public"])

cc_toolchain_suite(
	name = "custom_toolchain",
	toolchains = {
		"darwin|compiler": ":cc-compiler-darwin_x86_64",
	}
)

cc_library(
    name = "malloc",
)

cc_library(
    name = "stl",
)

filegroup(
    name = "empty",
    srcs = [],
)

filegroup(
    name = "cc_wrapper",
    srcs = ["cc_wrapper.sh"],
)

filegroup(
    name = "osx_tools_darwin_x86_64",
    srcs = [
        ":cc_wrapper",
        ":libtool",
        ":make_hashed_objlist.py",
        ":wrapped_clang",
        ":wrapped_clang_pp",
        ":wrapped_ar",
        ":xcrunwrapper.sh",
    ],
)

cc_toolchain(
    name = "cc-compiler-darwin_x86_64",
    all_files = ":osx_tools_darwin_x86_64",
    compiler_files = ":osx_tools_darwin_x86_64",
    cpu = "darwin",
    dwp_files = ":empty",
    dynamic_runtime_libs = [":empty"],
    linker_files = ":osx_tools_darwin_x86_64",
    objcopy_files = ":empty",
    static_runtime_libs = [":empty"],
    strip_files = ":osx_tools_darwin_x86_64",
    supports_param_files = 0,
)

CROSSTOOL:

major_version: "local"
minor_version: ""
default_target_cpu: "same_as_host"

default_toolchain {
  cpu: "darwin"
  toolchain_identifier: "local_darwin"
}

toolchain {
  abi_version: "local"
  abi_libc_version: "local"
  builtin_sysroot: ""
  compiler: "compiler"
  host_system_name: "local"
  needsPic: true
  target_libc: "macosx"
  target_cpu: "darwin"
  target_system_name: "local"
  toolchain_identifier: "local_darwin"

  tool_path { name: "ar" path: "/usr/bin/libtool" }
  tool_path { name: "compat-ld" path: "/usr/bin/ld" }
  tool_path { name: "cpp" path: "/usr/bin/cpp" }
  tool_path { name: "dwp" path: "/usr/bin/dwp" }
  tool_path { name: "gcc" path: "cc_wrapper.sh" }
  cxx_flag: "-std=c++0x"
  linker_flag: "-lstdc++"
  linker_flag: "-undefined"
  linker_flag: "dynamic_lookup"
  linker_flag: "-headerpad_max_install_names"
  # TODO(ulfjack): This is wrong on so many levels. Figure out a way to auto-detect the proper
  # setting from the local compiler, and also how to make incremental builds correct.
  cxx_builtin_include_directory: "/"
  tool_path { name: "gcov" path: "/usr/bin/gcov" }
  tool_path { name: "ld" path: "/usr/bin/ld" }
  tool_path { name: "nm" path: "/usr/bin/nm" }
  tool_path { name: "objcopy" path: "/usr/bin/objcopy" }
  objcopy_embed_flag: "-I"
  objcopy_embed_flag: "binary"
  tool_path { name: "objdump" path: "/usr/bin/objdump" }
  tool_path { name: "strip" path: "/usr/bin/strip" }

  unfiltered_cxx_flag: "-D__clang__"

  # Anticipated future default.
  unfiltered_cxx_flag: "-no-canonical-prefixes"

  # Make C++ compilation deterministic. Use linkstamping instead of these
  # compiler symbols.
  unfiltered_cxx_flag: "-Wno-builtin-macro-redefined"
  unfiltered_cxx_flag: "-D__DATE__=\"redacted\""
  unfiltered_cxx_flag: "-D__TIMESTAMP__=\"redacted\""
  unfiltered_cxx_flag: "-D__TIME__=\"redacted\""

  # Security hardening on by default.
  # Conservative choice; -D_FORTIFY_SOURCE=2 may be unsafe in some cases.
  compiler_flag: "-D_FORTIFY_SOURCE=1"
  compiler_flag: "-fstack-protector"

  # Enable coloring even if there's no attached terminal. Bazel removes the
  # escape sequences if --nocolor is specified.
  compiler_flag: "-fcolor-diagnostics"

  # All warnings are enabled. Maybe enable -Werror as well?
  compiler_flag: "-Wall"
  # Enable a few more warnings that aren't part of -Wall.
  compiler_flag: "-Wthread-safety"
  compiler_flag: "-Wself-assign"

  # Keep stack frames for debugging, even in opt mode.
  compiler_flag: "-fno-omit-frame-pointer"

  # Anticipated future default.
  linker_flag: "-no-canonical-prefixes"

  compilation_mode_flags {
    mode: DBG
    # Enable debug symbols.
    compiler_flag: "-g"
  }
  compilation_mode_flags {
    mode: OPT
    # No debug symbols.
    # Maybe we should enable https://gcc.gnu.org/wiki/DebugFission for opt or even generally?
    # However, that can't happen here, as it requires special handling in Bazel.
    compiler_flag: "-g0"

    # Conservative choice for -O
    # -O3 can increase binary size and even slow down the resulting binaries.
    # Profile first and / or use FDO if you need better performance than this.
    compiler_flag: "-O2"

    # Disable assertions
    compiler_flag: "-DNDEBUG"

    # Removal of unused code and data at link time (can this increase binary size in some cases?).
    compiler_flag: "-ffunction-sections"
    compiler_flag: "-fdata-sections"
  }
  linking_mode_flags { mode: DYNAMIC }
}

[matthias-kraft]

Hi @Timmmm thanks for sharing the CROSSTOOL and BUILD file, they worked for me and I can now happily run and build bazel projects with CLion (2017.2.3) on Mac (10.13).

Unfortunately, I cannot use the debugger yet. I encountered several errors trying to get it to work. First when clicking the debug symbol for a run configuration a small window with "unknown error" popped up. This was solved by adding a bazel before-run task in the run configuration as detailed here http://danieldanciu.blogspot.ch/2017/10/clion-with-bazel-unknown-error-when.html.

Then I received this "Unable to find Mach task port for process-id 57573: (os/kern) failure (0x5)".
(please check gdb is codesigned - see taskgated(8)). Which can be solved (or at least I think so) by following these steps https://gist.github.com/gravitylow/fb595186ce6068537a6e9da6d8b5b96d

After which I got an error message about NSUnbufferIO not being set followed by process error 255. I thought setting this variable in .bashrc to YES would help and indeed the error is gone.

Unfortunately, the debugger is still not working. Using lldb I click on debug and am left with process error 255 and nothing happening. Switching to the bundled gdb I get "During startup terminated with SIG113" and it suggests switching from gdb to lldb in the preferences. For gdb this is a known error and can be solved by adding "set startup-with-shell off" to ~/.gdbinit, but this seems to have no effect within CLion.

Does anyone have any ideas?

[Timmmm]

Good work @mkraft89. What was the variable you added to .bashrc?

According to this code exit code 255 happens when lldb-server gdbserver is run with no
extra arguments.

Although in my playing around with it, it actually prints the usage and calls exit(0). I might be being blind but I'm not sure how from the code. On the other hand it does not seem to be the highest quality code - just running this segfaults for me:

/Applications/Xcode.app/Contents/SharedFrameworks/LLDB.framework/Resources/lldb-server gdbserver --attach 123 foo

Maybe that is the startup-with-shell issue you mentioned? Maybe try running CLion under dtruss to see how it runs llvm-server?

[matthias-kraft]

@Timmmm. Thanks for helping out!
I added to the .bashrc export NSUnbufferedIO=YES. I will look into your suggestions, but it might be a few days before I get to it.

[RNabel]

With the most-recent published update and CLion 2017.3.3 this problem seems to be resolved on macOS 10.13.3 (for me at least).

[Traeyee]

2020 is coming, has this problem been solved?

[immars]

+1 for this issue.

[immars]

now my IDE is able to find std::string, but not std::time_t.

Anyone encountered the same issue?

[sgowroji]

Hi there! We're doing a clean up of old issues and will be closing this one. Please reopen if you’d like to discuss anything further. We’ll respond as soon as we have the bandwidth/resources to do so.

[cfstras]

FYI this is still an issue, the bazel plugin does not seem to support macOS.

[tpasternak]

Hey @cfstras could you share more details about the problem?

[tpasternak]

the last example shown in this thread seems to work better

[sgowroji]

Hello @cfstras, Could you please check the above comment and let us know if there is any update from your side. Thanks!

[cfstras]

Sorry, pretty stuffed week. I'll try to run another test soon.

[cfstras]

@sgowroji the minimal example seems to work fine now.
But even adding a simple library has problems: I've added spdlog to the minimal example (https://github.com/cfstras/clion-bazel).

• sync reports no issues
• Calling "run" from the BUILD file works
• Syntax highlighting sees errors
• hello.cc appears as "unsynced"

[tpasternak]

ok, I managed to reproduce this problem, but seems like something related to #4063 , which I hope will be merged soon.

The problem happens when you build something that is dependent on a different external dependencies set ten the current targets imported by Intellij. As a workaround you may just rerun the build manual

Screen.Recording.2023-02-06.at.11.43.54.mov

[barrylapthorn]

FWIW I had the same problem as the issue title on my Macbook

"Mac support: Bazel plugin doesn't work with C++ std lib and repositories"

The solution to my problem was that environment variables DEVELOPER_DIR and SDKROOT needed to be set, prior to launching CLion. The Bazel plugin (or possibly Bazel) does actually warn about them being unset in the Bazel plugin window in CLion, but doesn't suggest a fix, and a few hours of googling didn't find anything either.

[tpasternak]

@barrylapthorn but can you confirm the problem still exists?

[barrylapthorn]

@tpasternak - sorry for the slow reply. As long as those two env-vars are set, I do not see the problem. I suspect that it's something to do with how XCode installed itself on my Macbook (which I rarely use). I don't know if it's exactly the same problem as the OP, but it seems like it.

The warnings about the missing environment variables appear in the Bazel plugin's problem/issues pane. And since they're only a warning and in a different pane, they're easy to ignore even though they're the reason why the code resolution was failing (and in fact there was no code 'scope' for the particular file either).

It's entirely possible the warnings are in the cropped pane in the right hand corner of this post above: #109 (comment)

[tpasternak]

ok, it can be covered by #4651

[blorente]

That PR would indeed solve @barrylapthorn 's problem :D We saw the exact same symptoms.