Coverage runner: cannot locate file

[sgammon]

Hey there esteemed rules_apple authors,

When we are trying to get unit tests going with coverage using rules_apple and rules_swift, we're running into the following error:

(11:45:22) INFO: Invocation ID: edfea391-060a-458f-b218-aa9913badfe1
(11:45:22) INFO: Current date is 2021-04-17
(11:45:23) INFO: Analyzed target //ios:Tests (1 packages loaded, 11 targets configured).
(11:45:23) INFO: Found 1 test target...
(11:45:23) WARNING: /Volumes/.../ios/BUILD.bazel:144:14: runfiles symlink ios/Tests/SanityTest.swift -> ios/Tests/SanityTest.swift obscured by ios/Tests -> bazel-out/darwin-fastbuild/bin/.../ios/Tests
(11:45:25) FAIL: //ios:Tests (see /private/var/tmp/_bazel_sam.g/.../bazel-out/darwin-fastbuild/testlogs/.../ios/Tests/test.log)
(11:45:25) INFO: From Testing //ios:Tests:
==================== Test output for //ios:Tests:
+ [[ -z bazel-out/darwin-fastbuild/bin/.../Tests.instrumented_files ]]
+ export ROOT=/private/var/tmp/_bazel_sam.g/...
+ ROOT=/private/var/tmp/_bazel_sam.g/...
+ [[ bazel-out/darwin-fastbuild/bin/.../ios/Tests.instrumented_files != /* ]]
+ export COVERAGE_MANIFEST=/private/var/tmp/_bazel_sam.g/.../ios/Tests.instrumented_files
+ COVERAGE_MANIFEST=/private/var/tmp/_bazel_sam.g/.../ios/Tests.instrumented_files
+ export COVERAGE_DIR=bazel-out/darwin-fastbuild/testlogs/.../ios/Tests/_coverage
+ COVERAGE_DIR=bazel-out/darwin-fastbuild/testlogs/.../ios/Tests/_coverage
+ [[ bazel-out/darwin-fastbuild/testlogs/.../ios/Tests/_coverage == /private/var/tmp/_bazel_sam.g/...* ]]
+ COVERAGE_DIR=/private/var/tmp/_bazel_sam.g/.../ios/Tests/_coverage
+ mkdir -p /private/var/tmp/_bazel_sam.g/.../ios/Tests/_coverage
+ COVERAGE_OUTPUT_FILE=bazel-out/darwin-fastbuild/testlogs/.../ios/Tests/coverage.dat
+ [[ bazel-out/darwin-fastbuild/testlogs/.../ios/Tests/coverage.dat == /private/var/tmp/_bazel_sam.g/...* ]]
+ COVERAGE_OUTPUT_FILE=/private/var/tmp/_bazel_sam.g/.../ios/Tests/coverage.dat
+ export JAVA_COVERAGE_FILE=/private/var/tmp/_bazel_sam.g/.../ios/Tests/_coverage/jvcov.dat
+ JAVA_COVERAGE_FILE=/private/var/tmp/_bazel_sam.g/.../ios/Tests/_coverage/jvcov.dat
+ export COVERAGE=1
+ COVERAGE=1
+ export BULK_COVERAGE_RUN=1
+ BULK_COVERAGE_RUN=1
+ for name in '"$LCOV_MERGER"'
+ [[ ! -e '' ]]
+ echo --
--
+ echo Coverage runner: cannot locate file
Coverage runner: cannot locate file
+ exit 1
================================================================================
(11:45:25) ERROR: /Volumes/VANTAGE/platform/.../ios/BUILD.bazel:144:14: output 'ios/Tests/coverage.dat' was not created
(11:45:25) ERROR: /Volumes/VANTAGE/platform/.../ios/BUILD.bazel:144:14: Testing //ios:Tests failed: not all outputs were created or valid
Target //ios:Tests up-to-date:
  bazel-out/applebin_ios-ios_x86_64-fastbuild-ST-b9b93b70f384/bin/.../ios/Tests.xctest.dSYM/Contents/Resources/DWARF/Tests
  bazel-out/applebin_ios-ios_x86_64-fastbuild-ST-b9b93b70f384/bin/.../ios/Tests.xctest.dSYM/Contents/Info.plist
  dist/bin/.../ios/Tests
  bazel-out/applebin_ios-ios_x86_64-fastbuild-ST-b9b93b70f384/bin/.../ios/Tests.zip
(11:45:25) INFO: Elapsed time: 2.916s, Critical Path: 1.88s
(11:45:25) INFO: 18 processes: 10 internal, 7 darwin-sandbox, 1 local.
//ios:Tests                                                      FAILED in 0.1s
  /private/var/tmp/_bazel_sam.g/.../ios/Tests/test.log

Executed 1 out of 1 test: 1 fails locally.
(11:45:25) FAILED: Build did NOT complete successfully

Are we doing something wrong? Here are the relevant Bazel targets:

swift_test(
    name = "swift-tests",
    srcs = glob(["Tests/**/*.swift"]),
    swiftc_inputs = [":BridgingHeader.h"],
    copts = [
        "-import-objc-header", "$(rootpath :BridgingHeader.h)",
    ],
    deps = [
        ":SchemaModule",
        ":APIModule",
    ],
)

ios_unit_test(
    name = "Tests",
    deps = [":swift-tests"],
    minimum_os_version = "9.0",
)

SchemaModule is a swift_proto_library, and APIModule is a swift_grpc_library. Bazel is invoked with:

bazel  coverage --config=mac --verbose_failures --test_output=errors --combined_report=lcov   -- //ios:Tests

bazel.rc:

## iOS-related settings
build --experimental_use_llvm_covmap
build --apple_platform_type=ios
build --ios_simulator_version=14.2
build --ios_simulator_device="iPhone 12 Pro"
build --apple_generate_dsym=true
build --apple_bitcode=embedded
build --enable_apple_binary_native_protos=true

build:mac --define=platform=darwin
build:mac --copt -UDEBUG

[sgammon]

Additionally, when trying to run the Swift target directly (with the same settings above):

(11:54:20) ERROR: /Volumes/VANTAGE/platform/.../ios/BUILD.bazel:131:11: Linking ios/swift-tests failed: (Exit 1): cc_wrapper.sh failed: error executing command 
  (cd /private/var/tmp/_bazel_sam.g/4a5532b4122a2e552baae7c86b17fb6b/sandbox/darwin-sandbox/8499/execroot/BRICKTOP && \
  exec env - \
    APPLE_SDK_PLATFORM=MacOSX \
    APPLE_SDK_VERSION_OVERRIDE=11.0 \
    BAZEL_LINKLIBS=-l%:libstdc++.a \
    BAZEL_LINKOPTS=-lm \
    LOG_LEVEL=debug \
    PATH=/Users/sam.g/Library/Caches/bazelisk/downloads/bazelbuild/bazel-4.0.0-darwin-x86_64/bin:/Library/Java/JavaVirtualMachines/gvm-ee-jdk11/Contents/Home/bin:/Users/sam.g/Library/google-cloud-sdk/bin:/Users/sam.g/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/MacGPG2/bin:/Library/Apple/usr/bin:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/Users/sam.g/.cargo/bin:/Users/sam.g/Library/Android/sdk/tools:/Users/sam.g/Library/Android/sdk/platform-tools \
    XCODE_VERSION_OVERRIDE=12.2.0.12B45b \
  external/local_config_cc/cc_wrapper.sh @bazel-out/darwin-fastbuild/bin/.../ios/swift-tests-2.params)
Execution platform: @local_config_platform//:host

Use --sandbox_debug to see verbose messages from the sandbox cc_wrapper.sh failed: error executing command 
  (cd /private/var/tmp/_bazel_sam.g/4a5532b4122a2e552baae7c86b17fb6b/sandbox/darwin-sandbox/8499/execroot/BRICKTOP && \
  exec env - \
    APPLE_SDK_PLATFORM=MacOSX \
    APPLE_SDK_VERSION_OVERRIDE=11.0 \
    BAZEL_LINKLIBS=-l%:libstdc++.a \
    BAZEL_LINKOPTS=-lm \
    LOG_LEVEL=debug \
    PATH=/Users/sam.g/Library/Caches/bazelisk/downloads/bazelbuild/bazel-4.0.0-darwin-x86_64/bin:/Library/Java/JavaVirtualMachines/gvm-ee-jdk11/Contents/Home/bin:/Users/sam.g/Library/google-cloud-sdk/bin:/Users/sam.g/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/MacGPG2/bin:/Library/Apple/usr/bin:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/Users/sam.g/.cargo/bin:/Users/sam.g/Library/Android/sdk/tools:/Users/sam.g/Library/Android/sdk/platform-tools \
    XCODE_VERSION_OVERRIDE=12.2.0.12B45b \
  external/local_config_cc/cc_wrapper.sh @bazel-out/darwin-fastbuild/bin/.../ios/swift-tests-2.params)
Execution platform: @local_config_platform//:host

Use --sandbox_debug to see verbose messages from the sandbox
ld: warning: directory not found for option '-F/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.0.sdk/Developer/Library/Frameworks'
ld: building for macOS, but linking in object file built for iOS Simulator, file 'bazel-out/darwin-fastbuild/bin/.../ios/swift-tests_objs/Tests/SanityTest.swift.o' for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

[sgammon]

Worth noting, as well, that the .instrumented_files and related materials are filled out correctly.

[keith]

You can read a bit more about this problem here #691

For our use case we don't really care about this merging as much so we pass --test_env=LCOV_MERGER=/usr/bin/true to ignore this error

[sgammon]

@keith wouldn't that trample the LCOV merger for other coverage targets? we are also testing java, JS, etc, in the same run

nevertheless i'll give it a try; perhaps we can produce reports from each language target and merge ourselves.

[keith]

Ah yea if you ran that all in 1 invocation it would, alternatively I believe you can set that on the test runner itself if you specify a custom one like:

ios_test_runner(
    name = "custom_runner",
    test_environment = {"LCOV_MERGER": "/usr/bin/true"},
)

and then provided this to the runner attribute in your ios_unit_test target

[keith]

The coverage merger attribute has been merged here #1388 you can test with HEAD rules, but we should be good here, please reopen if you find more issues!