Argument list too long linking on mac #1637
Comments
|
oh, looking at that command line snippet, I bet it wasn't updating to rules_go 0.12 so much as adding an importmap on everything under still an issue, regardless... |
|
Looks like we'll need to switch to Will you need the fix for this backported to older release branches (e.g., 0.12), or is it okay for it to just be on 0.14? The fix will require the SDK to provide more information to the compiler than it currently does, so backporting will be non-trivial. Also, do you need the fix to support Go 1.8 or older? The feature that would fix this is in Go 1.9. Go 1.8 would require a very different solution. |
|
rules_go 0.14 should be fine. I don't think there's anything blocking us from updating besides time. The oldest version of Go we're using is 1.9, but that's also on a branch that's using a much older version of rules_go. The only branch encountering this issue is using go 1.10.3. |
|
thanks for filing @BenTheElder I completely forgot this morning https://gist.github.com/spiffxp/ef70004a6179fd4bf3efa720f111b038 if you want to see the actual command that failed otherwise, yes, what @ixdy said |
jayconrod
pushed a commit
to jayconrod/rules_go
that referenced
this issue
Aug 10, 2018
The Go toolchain has supported importcfg files since 1.9. These files give the build system finer control over dependencies using importmap and packagefile declarations. Using these files allows us to abandon -I and -L flags, which will help us stay under command line length limits. Fixes bazel-contrib#1637
jayconrod
pushed a commit
to jayconrod/rules_go
that referenced
this issue
Aug 10, 2018
The Go toolchain has supported importcfg files since 1.9. These files give the build system finer control over dependencies using importmap and packagefile declarations. Using these files allows us to abandon -I and -L flags, which will help us stay under command line length limits. Fixes bazel-contrib#1637
jayconrod
pushed a commit
to jayconrod/rules_go
that referenced
this issue
Aug 10, 2018
This was referenced Aug 10, 2018
jayconrod
added a commit
that referenced
this issue
Aug 13, 2018
The Go toolchain has supported importcfg files since 1.9. These files give the build system finer control over dependencies using importmap and packagefile declarations. Using these files allows us to abandon -I and -L flags, which will help us stay under command line length limits. Fixes #1637
|
Thanks @jayconrod ! |
jayconrod
pushed a commit
to jayconrod/rules_go
that referenced
this issue
Aug 13, 2018
|
FYI @BenTheElder @ixdy @spiffxp Please try with I think I'd like to tag a major release (0.15.0) with this soon (this week) rather than backport it to an older release branch. Let me know if that doesn't work for you. That may require Bazel 0.16.0 as a minimum supported version because of #1550, but that probably won't matter unless you're building for arm64. |
jayconrod
added a commit
that referenced
this issue
Aug 13, 2018
jayconrod
pushed a commit
that referenced
this issue
Aug 22, 2018
* Move legacy reproducibility test to new setup (#1585) * Move legacy reproducibility test to new setup Also included: 1. Adds an LLVM toolchain to the WORKSPACE file that can be used to manually test changes against clang on linux. 2. Adds os.PathSeparator to stripped absolute paths to be consistent with stripping being done everywhere else. 3. Adds a test to check the string "bazel-sandbox" in the binary. 4. Uses the cgo binary instead of a pure binary to check for reproducibility in the go_test. 5. Tags the target "collect_digests" as manual. To see how binaries are not reproducible with clang and `-g`, use ``` bazel test --copt=-g --crosstool_top=@llvm_toolchain//:toolchain \ //tests/reproducibility:go_default_test ``` * remove manual tag from reproducibility test * Do not use debug mode for reproducibility test. * Update bazel-toolchain to auto detect OS version * Add a primitive benchmark (#1599) bazel_benchmark checks out rules_go to a temporary directory, creates a temporary workspace, measures the time it takes to build various targets, then appends the times to a .csv file. This will probably be much more sophisticated in the future, but it's good to have something basic now. * go/tools/bazel_benchmark: extract some logic into bash script (#1601) bazel_benchmark.sh is now responsible for cloning rules_go at master into a temp directory. This script can be copied to a bin directory and run with a timer. The rest of the bazel_benchmark.go logic will run at the tip of master. Also: record Bazel version in the output file. * Set Bazel version tested in Travis CI to 0.15.0 (#1604) * Speed up downloading of @go_googleapis by using http_archive. (#1603) The googleapis repository seems to be of such a size that it takes a long time to clone on a link with lower bandwidth. So long, in fact, that it causes timeouts in my case. * Use add_all(), add_joined() instead of deprecated functionality of add() (#1602) * Change crosstool dependency to @bazel_tools//tools/cpp:current_cc_toolchain (#1605) //tools/defaults is a special case which is being removed in Bazel. * Add go_sdk rule and GoSDK provider (#1606) go_sdk is a new rule that gathers information about an SDK and returns a GoSDK provider which will get wired into the toolchain. package_list is a new rule that generates a list of importable packages from the sources in the SDK (previously, we invoked go list, which is slower). * Wire go_sdk and go_toolchain together (#1607) * go_toolchain has a new mandatory attribute, "sdk", which be something that provides GoSDK. * go_host_sdk, go_local_sdk, and go_download_sdk are now macros that wrap the old rules. Each rule declares toolchains in its BUILD.bazel file that work on the host architecture. The macro calls register_toolchains with these. * go_register_toolchains no longer calls register_toolchains, but it will an SDK rule if "go_sdk" isn't defined. This is a step toward allowing multiple SDKs to support multiple execution platforms. * Action inputs are narrowed to use go.sdk.tools and go.stdlib.libs rather than larger sets of files. * stdlib now uses precompiled libraries if the mode is compatible (#1608) * Remove deprecated --batch flag (#1617) * Add go_wrap_sdk rule (#1618) go_wrap_sdk allows you to configure a Go SDK that was downloaded or located with another repository rule. Related #1611 * Optimize args and inputs construction (#1610) * Use add_all() to lazily construct args * add_joined() omits the argument if value is an empty list * Optimize compile * Optimize cover * Simplify tags argument construction * Use any() instead of a dict * Undo depset() usage * Statically link tool binaries (#1615) * A few arguments construction cleanups (#1621) * Update toolchain and provider documentation [skip ci] (#1622) * Document race, msan and other attributes for go_binary, go_test [skip ci] (#1624) * Create .bazelrc (#1626) * Create .bazelrc See bazelbuild/bazel#5756 (comment) * Update .bazelrc * Remove explicit Label() construction (#1627) attr.label() converts strings into labels by itself. * Make go_sdk's package_list optional (#1625) This will eventually be removed when old versions of Gazelle are no longer supported and nothing depends on the file by name. If not provided as an input, go_sdk will generate the file itself. * Update deprecated single_file -> allow_single_file attribute (#1628) Also remove allow_files where it conflicts with allow_single_file (not allowed). Also remove both allow_files and allow_single_file in private attributes where executable is set to True (a file cannot be specified anyway - private attribute). * Document how to avoid proto conflicts [skip ci] (#1631) Fixes #1548 * Set RULES_GO_VERSION to 0.14.0 (#1633) * Propagate mode aspect on "_coverdata" edges (#1632) * Propagate mode aspect on "_coverdata" edges This ensures the coverdata library is built in the same mode as the binary that depends on it. Fixes #1630 * set pure = "on" on test to make CI happy * Update dependencies (#1634) bazel_gazelle to master as of 2018-08-06 com_google_protobuf to v3.6.1 com_github_goog_protobuf to v1.1.1 org_golang_x_net to master as of 2018-08-06 org_golang_google_grpc to v1.14.0 org_golang_google_genproto to master as of 2018-08-06 go_googleapis to master as of 2018-08-06 com_github_kevinburke_go_bindata to v3.11.0 org_golang_x_tools to master as of 2018-08-07 * Announce release 0.14.0 [skip ci] (#1636) Also, fix gazelle example to use prefix directive instead of attribute. * Add CI config to test on RBE. (#1638) * Add CI config to test on RBE. * Disable BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN. This is set by default in the rbe_ubuntu1604 platform, but tests in this repo need this to be disabled. * Skip tests that are not RBE compatible. * Use latest release version of bazel-toolchains repo. (#1639) * Declare org_golang_x_sys in go_rules_dependencies (#1649) The newest version of gRPC depends on @org_golang_x_sys. Since we provide other gRPC dependencies, we should declare this one as well. Fixes #1648 * Document how to override go_rules_dependencies [skip ci] (#1650) Related #1649 * Update minimum version of Bazel for Travis CI to 0.16.0 (#1651) Also, remove logic in .travis.yml for downloading Bazel at HEAD. We're not doing that anymore. * Actions that use go.args may now use param files automatically (#1652) Multiple param files are now supported as well. * Split go.args into go.builder_args and go.tool_args (#1653) Both helpers enable multiline files. Any action using either of these helpers should support them. Other actions may use go.actions.args. go.builder_args adds default arguments that builders should be able to interpret, including -sdk and -tags. go.args is deprecated. * Use -importcfg files for compiling and linking (#1654) The Go toolchain has supported importcfg files since 1.9. These files give the build system finer control over dependencies using importmap and packagefile declarations. Using these files allows us to abandon -I and -L flags, which will help us stay under command line length limits. Fixes #1637 * Update genproto dependencies (#1657) * update org_golang_google_genproto * update go_googleapis * Add test generates long compile/link command lines (#1655) Related #1637 * Remove 'cfg = "data"' from all attributes (#1658) The "data" configuration has been deprecated for a while and has no effect. * Remove gazelle and its deps from go_rules_dependencies (#1659) go_rules_dependencies no longer declares the following repositories: * bazel_gazelle * com_github_bazelbuild_buildtools * com_github_pelletier_go_toml The "gazelle" rule is removed from //go:def.bzl. It has been deprecated for some time, and "gazelle fix" replaces it. * Windows: Use absolute and shortened path for GOROOT environment variable (#1647) * Update tests ahead of Go 1.11 (#1661) A test in the old version of org_golang_x_crypto we were testing fails with Go 1.11. This is fixed in newer versions. * Remove uses of deprecated dictionary concatenation (#1663) This removes the need for --incompatible_disallow_dict_plus * Force absolute paths in builders (#1664) * Update org_golang_x_tools to master as of 2018-08-15 (#1662) * Set RULES_GO_VERSION to 0.15.0 (#1665) * Announce release 0.15.0 [skip ci] (#1666) * one character fix to README boilerplate [skip ci] (#1667) * doc: fix grammatical error (#1671)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
On kubernetes master branch
bazel build //test/e2e:e2e.testfails with:Action failed to execute: java.io.IOException: Cannot run program "/usr/bin/sandbox-exec" (in directory "/private/var/tmp/_bazel_bentheelder/c99a7ed2e9dbc7676356495197988838/sandbox/darwin-sandbox/1/execroot/__main__"): error=7, Argument list too long(above this is a massive command line with lots of args like
-dep '//staging/src/k8s.io/apimachinery/pkg/api/meta/table:go_default_library=k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/api/meta/table=bazel-out/darwin-fastbuild/bin/staging/src/k8s.io/apimachinery/pkg/api/meta/table/darwin_amd64_stripped/go_default_library~/k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/api/meta/table.a'It looks like this is a massive link option, which become to large after updating to rules go 0.12
1ad1c8c7f8Merge pull request #64122 from ixdy/update-rules_go-and-gazelleseems to be the breaking point./cc @ixdy
reported by @spiffxp
The text was updated successfully, but these errors were encountered: