save some CI diskspace, use ubuntu 20.04 and later libgit2 #693
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This was referenced Jul 26, 2023
This was referenced Jul 29, 2023
analytics.test.ts was trying to call the real gh cli rather than a mock, which was causing errors when gh not installed, and process and handle leaks when gh was installed. Fixed bu using mock execa as we don't need real gh to test the analytics tracking Jest was also reporting leaks of timeouts after above fix, so added dispose() to debounce.ts and call dispose() from githubCodeReviewProvider Found shared/ depends on reviewstack generated textmate grammars, so can't run concurrently without flakiness (fails with missing import). Fixed test ordering in verify-addons-folder.py Tests also log about missing watchman which triggers a failure due to logs after test exit, even if watchman disabled. Install fb-watchman to silence it Tes plan: Before, ./verify-addons-folder.py tests broken in CI with: ``` Error: Unhandled error. (Error: GhNotInstalledError: Error: Command failed with ENOENT: gh api graphql -f searchQuery=repo:facebook/sapling is:pr author:@me -F numToFetch=100 --hostname github.com -f query= query YourPullRequestsQuery($searchQuery: String!, $numToFetch: Int!) { search(query: $searchQuery, type: ISSUE, first: $numToFetch) { nodes { ... on PullRequest { ``` ``` After, works
fix two problems to causing build errors on mononoke macOS build. For mononoke linux the next change in the stack to fix linking is also needed
1) mononoke uses anyhow's backtrace feature. anyhow 1.0.74 changed its build.rs detection for backtrace which no longer finds it in the the "stable compiler plus RUSTC_BOOTSTRAP set" combination
For now fix cargo by pinning the Cargo build to the existing 1.0.71 requirement.
2) rust 1.72 removed the hash_drain_filter feature, which causes the build to start to fail. Removed the usage.
Test plan:
mononoke macOS CI, goes from broken to working
Build locally on macOS with `./build/fbcode_builder/getdeps.py --allow-system-packages build --src-dir=. mononoke`
Before, fails:
```
error[E0599]: no method named `backtrace` found for struct `Arc<anyhow::Error>` in the current scope
--> megarepo_api/megarepo_error/src/lib.rs:33:24
|
33 | self.0.backtrace()
| ^^^^^^^^^ method not found in `Arc<Error>`
...
61 | cloneable_error!(InternalError);
| ------------------------------- in this macro invocation
```
and on rust 1.72 failed with
```
Command '['cargo', 'build', '--workspace', '-j2', '--out-dir', '/tmp/fbcode_builder_getdeps-ZhomeZrunnerZworkZsaplingZsaplingZbuildZfbcode_builder/installed/mononoke/bin', '-Zunstable-options']' returned non-zero exit status 101.
!! Failed
error[E0635]: unknown feature `hash_drain_filter`
--> cmdlib/sharding/src/lib.rs:8:12
|
8 | #![feature(hash_drain_filter)]
| ^^^^^^^^^^^^^^^^^
```
After, works
During my recent reviewstack changes I noticed that mononoke links fine on macOS, but github PRs are failing on linux with duplicate zstd symbols during linking. To fix this, switch to lld for the final link on linux by setting the cargo options, which needs a getdeps cargo.py change to ensure they aren't overwritten It works locally but looks like its running our of disk space on github. Next change in stack addresses github CI diskspace Test plan: Install the new lld system dep with `sudo ./build/fbcode_builder/getdeps.py install-system-deps --recursive mononoke` Run a build locally with `./build/fbcode_builder/getdeps.py --allow-system-packages build --src-dir=. mononoke` Before, zstd link errors like https://github.com/facebook/sapling/actions/runs/5432912652/jobs/9880272333: ``` /usr/bin/ld: /home/alex/local/tmp/toolbox/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/build/mononoke/debug/deps/libzstd_sys-837ebc89eb1d31b6.rlib(zstd_lazy.o): in function `ZSTD_compressBlock_btlazy2_extDict': zstd_lazy.c:(.text.ZSTD_compressBlock_btlazy2_extDict+0x0): multiple definition of `ZSTD_compressBlock_btlazy2_extDict'; /home/alex/local/tmp/toolbox/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/build/mononoke/debug/deps/libzstd_legacy_mononoke_sys-78a66a56bd363eb2.rlib(zstd_lazy.o):zstd_lazy.c:(.text.ZSTD_compressBlock_btlazy2_extDict+0x0): first defined here collect2: error: ld returned 1 exit status ``` After, works: ``` Compiling mononoke v0.1.0 (/home/alex/local/tmp/toolbox/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/build/mononoke/source/eden/mononoke/server) Finished dev [unoptimized] target(s) in 1m 17s ```
Allow getdeps to free up some disk from the runner and intermediate build steps as some runs (notably the linux eden and mononoke ones) are hitting disk space limits Test Plan: Regenerate and run CI for mononoke. Mononoke Mac should continue to work Mononoke Linux should go from broken to working ``` ./build/fbcode_builder/getdeps.py --allow-system-packages generate-github-actions --free-up-disk --src-dir=. --output-dir=.github/workflows --job-name="Mononoke " --job-file-prefix=mononoke_ mononoke ```
Generated actions files created from getdeps.py were missing rust toolchain which meant annoying manual edits This change also makes the github actions generation honor the --no-tests argument, which is useful when regenerating the edenfs actions Test plan: To test we add the missing undeclared rust-shed dependency to the eden manifest and regenerate. Generate updated actions yaml with: ``` ./build/fbcode_builder/getdeps.py --allow-system-packages generate-github-actions --no-tests --free-up-disk --os-type=linux --src-dir=. --output-dir=.github/workflows --job-name "EdenFS " --job-file-prefix=edenfs_ eden ``` Before, was missing the rust toolchain in the auto regenerated eden actions file. After, toolchain is present Before, was missing the --no-tests in the "Build eden" step of the auto-regenerated eden actions file. After, has it Eden stil needs some other fixes to build, which follow in this stqck
edenfs was missing a declared dep on edencommon in the rpc code CMakeLists.txt, this fixed missing edencommon headers
Test plan:
Build locally: `./build/fbcode_builder/getdeps.py --allow-system-packages build --src-dir=. eden`
Before:
```
FAILED: eden/fs/nfs/portmap/CMakeFiles/eden_nfs_portmap_server.dir/Rpcbindd.cpp.o
/usr/bin/c++ -DBOOST_ALL_NO_LIB -DFOLLY_XLOG_STRIP_PREFIXES=\"/home/alex/local/sapling:/home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/build/eden\" -DGFLAGS_IS_A_DLL=0 -I. -I/home/alex/local/sapling -I/home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/fb303/include -I/home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/fizz/include -I/home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/wangle/include -I/home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/fbthrift/include -I/home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/blake3-lrolsLBeklWWBLYfMM4fF2TJGwCeReSgHBUbEgA3TFw/include -isystem /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/folly/include -isystem /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/fmt-_ZorQJC2eeu9K7XicUjMKUe54C8sh6ZsaDZIZfQ_QmE/include -g -Wall -Wextra -Wno-deprecated -Wno-deprecated-declarations -Wno-nullability-completeness -Wno-unknown-pragmas -O2 -g -DNDEBUG -pthread -std=gnu++17 -MD -MT eden/fs/nfs/portmap/CMakeFiles/eden_nfs_portmap_server.dir/Rpcbindd.cpp.o -MF eden/fs/nfs/portmap/CMakeFiles/eden_nfs_portmap_server.dir/Rpcbindd.cpp.o.d -o eden/fs/nfs/portmap/CMakeFiles/eden_nfs_portmap_server.dir/Rpcbindd.cpp.o -c /home/alex/local/sapling/eden/fs/nfs/portmap/Rpcbindd.cpp
In file included from /home/alex/local/sapling/eden/fs/nfs/portmap/Rpcbindd.h:14,
from /home/alex/local/sapling/eden/fs/nfs/portmap/Rpcbindd.cpp:8:
/home/alex/local/sapling/eden/fs/utils/PathFuncs.h:24:10: fatal error: eden/common/utils/StringConv.h: No such file or directory
24 | #include "eden/common/utils/StringConv.h"
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
```
After, works and gets to the next error, see next in stack
Summary:
Update the googletest manifest as ubuntu's LTS packages are too old. 22.10 on looks fine
Test Plan:
Build locally with ./build/fbcode_builder/getdeps.py --allow-system-packages build --src-dir=. eden
before, breaks at:
```
/usr/bin/c++ -DBOOST_ALL_NO_LIB -DFOLLY_XLOG_STRIP_PREFIXES=\"/home/alex/local/sapling:/home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/build/eden\" -DGFLAGS_IS_A_DLL=0 -I. -I/home/alex/local/sapling -I/home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/blake3-lrolsLBeklWWBLYfMM4fF2TJGwCeReSgHBUbEgA3TFw/include -isystem /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/folly/include -isystem /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/fb303/include -isystem /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/fizz/include -isystem /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/wangle/include -isystem /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/fbthrift/include -isystem /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/fmt-_ZorQJC2eeu9K7XicUjMKUe54C8sh6ZsaDZIZfQ_QmE/include -isystem /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/mvfst/include -isystem /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/edencommon/include -isystem /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/cpptoml-_zeUSHY925tqbka8WnSRFSO5ZQbE56aQYsrCa6wv7mU/include -isystem /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/rocksdb-Y6wy710FNBv6sNEe4kCJ_-a3O5-tys4ivojRziE0GyM/include -g -Wall -Wextra -Wno-deprecated -Wno-deprecated-declarations -Wno-nullability-completeness -Wno-unknown-pragmas -O2 -g -DNDEBUG -fsized-deallocation -pthread -std=gnu++17 -MD -MT eden/fs/utils/test/CMakeFiles/eden_utils_test.dir/SourceLocationTest.cpp.o -MF eden/fs/utils/test/CMakeFiles/eden_utils_test.dir/SourceLocationTest.cpp.o.d -o eden/fs/utils/test/CMakeFiles/eden_utils_test.dir/SourceLocationTest.cpp.o -c /home/alex/local/sapling/eden/fs/utils/test/SourceLocationTest.cpp
In file included from /usr/include/gmock/gmock-spec-builders.h:75,
from /usr/include/gmock/gmock-generated-function-mockers.h:47,
from /usr/include/gmock/gmock-function-mocker.h:39,
from /usr/include/gmock/gmock.h:61,
from /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/folly/include/folly/portability/GMock.h:32,
from /home/alex/local/sapling/eden/fs/utils/test/SourceLocationTest.cpp:11:
/usr/include/gmock/gmock-matchers.h: In instantiation of ‘bool testing::internal::HasSubstrMatcher<StringType>::MatchAndExplain(const MatcheeStringType&, testing::MatchResultListener*) const [with MatcheeStringType = std::basic_string_view<char>; StringType = std::__cxx11::basic_string<char>]’:
/usr/include/gtest/gtest-matchers.h:484:47: required from ‘bool testing::PolymorphicMatcher<Impl>::MonomorphicImpl<T>::MatchAndExplain(T, testing::MatchResultListener*) const [with T = const std::basic_string_view<char>&; Impl = testing::internal::HasSubstrMatcher<std::__cxx11::basic_string<char> >]’
/usr/include/gtest/gtest-matchers.h:483:18: required from here
/usr/include/gmock/gmock-matchers.h:760:23: error: invalid initialization of reference of type ‘const std::__cxx11::basic_string<char>&’ from expression of type ‘const std::basic_string_view<char>’
760 | const StringType& s2(s);
```
after, works on ubuntu 20.04. On 22.04 gets to the next problem, see the ImmediateFuture.h change later in this stack
GCC complained about these redundant moves, spamming the logs.
Test Plan:
Build locally with ./build/fbcode_builder/getdeps.py --allow-system-packages build --src-dir=. eden
before, warnings:
```
/home/runner/work/sapling/sapling/eden/fs/utils/ImmediateFuture-inl.h:356:25: warning: redundant move in return statement [-Wredundant-move]
356 | return std::move(res);
| ^
/home/runner/work/sapling/sapling/eden/fs/utils/ImmediateFuture-inl.h:356:25: note: remove ‘std::move’ call
/home/runner/work/sapling/sapling/eden/fs/utils/ImmediateFuture-inl.h: In instantiation of ‘facebook::eden::ImmediateFuture<std::vector<folly::Try<T> > > facebook::eden::collectAll(std::vector<facebook::eden::ImmediateFuture<T> >) [with T = {anonymous}::ImmediateFuture_collectUncopyable_Test::TestBody()::Uncopyable]’:
/home/runner/work/sapling/sapling/eden/fs/utils/test/ImmediateFutureTest.cpp:350:39: required from here
/home/runner/work/sapling/sapling/eden/fs/utils/ImmediateFuture-inl.h:356:25: warning: redundant move in return statement [-Wredundant-move]
/home/runner/work/sapling/sapling/eden/fs/utils/ImmediateFuture-inl.h:356:25: note: remove ‘std::move’ call
/home/runner/work/sapling/sapling/eden/fs/utils/ImmediateFuture-inl.h: In instantiation of ‘facebook::eden::ImmediateFuture<typename facebook::eden::detail::continuation_result<typename std::enable_if<is_invocable_v<Func, folly::Try<T> >, void>::type, Func, folly::Try<T> >::type> facebook::eden::ImmediateFuture<T>::thenTry(Func&&) && [with Func = facebook::eden::ImmediateFuture<T>::ensure(Func&&) && [with Func = {anonymous}::ImmediateFuture_ensureBasic_Test::TestBody()::<lambda()>&; T = int]::<lambda(folly::Try<int>)>; T = int; typename facebook::eden::detail::continuation_result<typename std::enable_if<is_invocable_v<Func, folly::Try<T> >, void>::type, Func, folly::Try<T> >::type = int]’:
/home/runner/work/sapling/sapling/eden/fs/utils/ImmediateFuture-inl.h:196:8: required from ‘facebook::eden::ImmediateFuture<T> facebook::eden::ImmediateFuture<T>::ensure(Func&&) && [with Func = {anonymous}::ImmediateFuture_ensureBasic_Test::TestBody()::<lambda()>&; T = int]’
/home/runner/work/sapling/sapling/eden/fs/utils/test/ImmediateFutureTest.cpp:52:41: required from here
/home/runner/work/sapling/sapling/eden/fs/utils/ImmediateFuture-inl.h:240:31: warning: redundant move in return statement [-Wredundant-move]
240 | return std::move(semiFut);
```
after, they are gone!
Save some time and diskspace in CI builds for github by using the system libraries Test plan: Install deps and build locally with ``` sudo ./build/fbcode_builder/getdeps.py --allow-system-packages install-system-deps --recursive eden ... ./build/fbcode_builder/getdeps.py --allow-system-packages build --src-dir=. eden ``` before: we build libgit2 from source after: we use the system library
|
@genevievehelsel has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
facebook-github-bot
pushed a commit
that referenced
this pull request
Oct 5, 2023
Summary:
reviewstack: fix verify addons/ folder github ci
analytics.test.ts was trying to call the real gh cli rather than a mock, which was causing errors when gh not installed, and process and handle leaks when gh was installed. Fixed bu using mock execa as we don't need real gh to test the analytics tracking
Jest was also reporting leaks of timeouts after above fix, so added dispose() to debounce.ts and call dispose() from githubCodeReviewProvider
Found shared/ depends on reviewstack generated textmate grammars, so can't run concurrently without flakiness (fails with missing import). Fixed test ordering in verify-addons-folder.py
Tests also log about missing watchman which triggers a failure due to logs after test exit, even if watchman disabled. Install fb-watchman to silence it
Tes plan:
Before, ./verify-addons-folder.py tests broken in CI with:
```
Error: Unhandled error. (Error: GhNotInstalledError: Error: Command failed with ENOENT: gh api graphql -f searchQuery=repo:facebook/sapling is:pr author:me -F numToFetch=100 --hostname github.com -f query=
query YourPullRequestsQuery($searchQuery: String!, $numToFetch: Int!) {
search(query: $searchQuery, type: ISSUE, first: $numToFetch) {
nodes {
... on PullRequest {
```
```
After, works
---
Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/facebook/sapling/pull/730).
* #693
* #696
* #692
* #691
* #682
* #689
* #697
* #706
* __->__ #730
Pull Request resolved: #730
Reviewed By: evangrayk
Differential Revision: D49875301
Pulled By: genevievehelsel
fbshipit-source-id: 2e322467200189781b881decc3a3af5647605cc6
genevievehelsel
pushed a commit
to genevievehelsel/velox
that referenced
this pull request
Oct 5, 2023
Summary: fix mononoke linux local cargo build with lld During my recent reviewstack changes I noticed that mononoke links fine on macOS, but github PRs are failing on linux with duplicate zstd symbols during linking. To fix this, switch to lld for the final link on linux by setting the cargo options, which needs a getdeps cargo.py change to ensure they aren't overwritten It works locally but looks like its running our of disk space on github. Next change in stack addresses github CI diskspace X-link: facebook/sapling#697 Test Plan: Install the new lld system dep with `sudo ./build/fbcode_builder/getdeps.py install-system-deps --recursive mononoke` Run a build locally with `./build/fbcode_builder/getdeps.py --allow-system-packages build --src-dir=. mononoke` Before, zstd link errors like https://github.com/facebook/sapling/actions/runs/5432912652/jobs/9880272333: ``` /usr/bin/ld: /home/alex/local/tmp/toolbox/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/build/mononoke/debug/deps/libzstd_sys-837ebc89eb1d31b6.rlib(zstd_lazy.o): in function `ZSTD_compressBlock_btlazy2_extDict': zstd_lazy.c:(.text.ZSTD_compressBlock_btlazy2_extDict+0x0): multiple definition of `ZSTD_compressBlock_btlazy2_extDict'; /home/alex/local/tmp/toolbox/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/build/mononoke/debug/deps/libzstd_legacy_mononoke_sys-78a66a56bd363eb2.rlib(zstd_lazy.o):zstd_lazy.c:(.text.ZSTD_compressBlock_btlazy2_extDict+0x0): first defined here collect2: error: ld returned 1 exit status ``` After, works: ``` Compiling mononoke v0.1.0 (/home/alex/local/tmp/toolbox/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/build/mononoke/source/eden/mononoke/server) Finished dev [unoptimized] target(s) in 1m 17s ``` --- Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/facebook/sapling/pull/697). * facebook/sapling#693 * facebook/sapling#696 * facebook/sapling#692 * facebook/sapling#691 * facebook/sapling#682 * facebook/sapling#689 * __->__ facebook/sapling#697 * facebook/sapling#706 * facebook/sapling#730 Differential Revision: D49875277 Pulled By: genevievehelsel
facebook-github-bot
pushed a commit
to facebookincubator/velox
that referenced
this pull request
Oct 7, 2023
Summary: Pull Request resolved: #6923 fix mononoke linux local cargo build with lld During my recent reviewstack changes I noticed that mononoke links fine on macOS, but github PRs are failing on linux with duplicate zstd symbols during linking. To fix this, switch to lld for the final link on linux by setting the cargo options, which needs a getdeps cargo.py change to ensure they aren't overwritten It works locally but looks like its running our of disk space on github. Next change in stack addresses github CI diskspace X-link: facebook/sapling#697 Test Plan: Install the new lld system dep with `sudo ./build/fbcode_builder/getdeps.py install-system-deps --recursive mononoke` Run a build locally with `./build/fbcode_builder/getdeps.py --allow-system-packages build --src-dir=. mononoke` Before, zstd link errors like https://github.com/facebook/sapling/actions/runs/5432912652/jobs/9880272333: ``` /usr/bin/ld: /home/alex/local/tmp/toolbox/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/build/mononoke/debug/deps/libzstd_sys-837ebc89eb1d31b6.rlib(zstd_lazy.o): in function `ZSTD_compressBlock_btlazy2_extDict': zstd_lazy.c:(.text.ZSTD_compressBlock_btlazy2_extDict+0x0): multiple definition of `ZSTD_compressBlock_btlazy2_extDict'; /home/alex/local/tmp/toolbox/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/build/mononoke/debug/deps/libzstd_legacy_mononoke_sys-78a66a56bd363eb2.rlib(zstd_lazy.o):zstd_lazy.c:(.text.ZSTD_compressBlock_btlazy2_extDict+0x0): first defined here collect2: error: ld returned 1 exit status ``` After, works: ``` Compiling mononoke v0.1.0 (/home/alex/local/tmp/toolbox/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/build/mononoke/source/eden/mononoke/server) Finished dev [unoptimized] target(s) in 1m 17s ``` --- Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/facebook/sapling/pull/697). * facebook/sapling#693 * facebook/sapling#696 * facebook/sapling#692 * facebook/sapling#691 * facebook/sapling#682 * facebook/sapling#689 * __->__ facebook/sapling#697 * facebook/sapling#706 * facebook/sapling#730 Reviewed By: sggutier Differential Revision: D49875277 Pulled By: genevievehelsel fbshipit-source-id: 998f525fd71333b31747d0ad0e0dda3ebe2b0796
facebook-github-bot
pushed a commit
to facebookincubator/velox
that referenced
this pull request
Oct 7, 2023
Summary: Pull Request resolved: #6927 allow getdeps github actions to free up disk Allow getdeps to free up some disk from the runner and intermediate build steps as some runs (notably the linux eden and mononoke ones) are hitting disk space limits X-link: facebook/sapling#689 Test Plan: Regenerate and run CI for mononoke. Mononoke Mac should continue to work Mononoke Linux should go from broken to working ``` ./build/fbcode_builder/getdeps.py --allow-system-packages generate-github-actions --free-up-disk --src-dir=. --output-dir=.github/workflows --job-name="Mononoke " --job-file-prefix=mononoke_ mononoke ``` --- Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/facebook/sapling/pull/689). * facebook/sapling#693 * facebook/sapling#696 * facebook/sapling#692 * facebook/sapling#691 * facebook/sapling#682 * __->__ facebook/sapling#689 * facebook/sapling#697 * facebook/sapling#706 * facebook/sapling#730 Reviewed By: sggutier Differential Revision: D49875256 Pulled By: genevievehelsel fbshipit-source-id: b85b6b2f11857670915b64f47d3c0abd4ca8ca31
facebook-github-bot
pushed a commit
to facebookincubator/velox
that referenced
this pull request
Oct 7, 2023
Summary: Pull Request resolved: #6924 fix getdeps actions generation for rust toolchain Generated actions files created from getdeps.py were missing rust toolchain which meant annoying manual edits This change also makes the github actions generation honor the --no-tests argument, which is useful when regenerating the edenfs actions X-link: facebook/sapling#682 Test Plan: To test we add the missing undeclared rust-shed dependency to the eden manifest and regenerate. Generate updated actions yaml with: ``` ./build/fbcode_builder/getdeps.py --allow-system-packages generate-github-actions --no-tests --free-up-disk --os-type=linux --src-dir=. --output-dir=.github/workflows --job-name "EdenFS " --job-file-prefix=edenfs_ eden ``` Before, was missing the rust toolchain in the auto regenerated eden actions file. After, toolchain is present Before, was missing the --no-tests in the "Build eden" step of the auto-regenerated eden actions file. After, has it Eden stil needs some other fixes to build, which follow in this stqck --- Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/facebook/sapling/pull/682). * facebook/sapling#693 * facebook/sapling#696 * facebook/sapling#692 * facebook/sapling#691 * __->__ facebook/sapling#682 * facebook/sapling#689 * facebook/sapling#697 * facebook/sapling#706 * facebook/sapling#730 Reviewed By: sggutier Differential Revision: D49875258 Pulled By: genevievehelsel fbshipit-source-id: 173f86083ba92bab4063813d3e392df428b9ffe4
facebook-github-bot
pushed a commit
to facebookincubator/velox
that referenced
this pull request
Oct 7, 2023
Summary: Pull Request resolved: #6926 fix edenfs oss build, ubuntu googletest is too old Update the googletest manifest as ubuntu's LTS packages are too old. 22.10 on looks fine X-link: facebook/sapling#692 Test Plan: Build locally with ./build/fbcode_builder/getdeps.py --allow-system-packages build --src-dir=. eden before, breaks at: ``` /usr/bin/c++ -DBOOST_ALL_NO_LIB -DFOLLY_XLOG_STRIP_PREFIXES=\"/home/alex/local/sapling:/home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/build/eden\" -DGFLAGS_IS_A_DLL=0 -I. -I/home/alex/local/sapling -I/home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/blake3-lrolsLBeklWWBLYfMM4fF2TJGwCeReSgHBUbEgA3TFw/include -isystem /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/folly/include -isystem /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/fb303/include -isystem /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/fizz/include -isystem /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/wangle/include -isystem /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/fbthrift/include -isystem /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/fmt-_ZorQJC2eeu9K7XicUjMKUe54C8sh6ZsaDZIZfQ_QmE/include -isystem /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/mvfst/include -isystem /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/edencommon/include -isystem /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/cpptoml-_zeUSHY925tqbka8WnSRFSO5ZQbE56aQYsrCa6wv7mU/include -isystem /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/rocksdb-Y6wy710FNBv6sNEe4kCJ_-a3O5-tys4ivojRziE0GyM/include -g -Wall -Wextra -Wno-deprecated -Wno-deprecated-declarations -Wno-nullability-completeness -Wno-unknown-pragmas -O2 -g -DNDEBUG -fsized-deallocation -pthread -std=gnu++17 -MD -MT eden/fs/utils/test/CMakeFiles/eden_utils_test.dir/SourceLocationTest.cpp.o -MF eden/fs/utils/test/CMakeFiles/eden_utils_test.dir/SourceLocationTest.cpp.o.d -o eden/fs/utils/test/CMakeFiles/eden_utils_test.dir/SourceLocationTest.cpp.o -c /home/alex/local/sapling/eden/fs/utils/test/SourceLocationTest.cpp In file included from /usr/include/gmock/gmock-spec-builders.h:75, from /usr/include/gmock/gmock-generated-function-mockers.h:47, from /usr/include/gmock/gmock-function-mocker.h:39, from /usr/include/gmock/gmock.h:61, from /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/folly/include/folly/portability/GMock.h:32, from /home/alex/local/sapling/eden/fs/utils/test/SourceLocationTest.cpp:11: /usr/include/gmock/gmock-matchers.h: In instantiation of ‘bool testing::internal::HasSubstrMatcher<StringType>::MatchAndExplain(const MatcheeStringType&, testing::MatchResultListener*) const [with MatcheeStringType = std::basic_string_view<char>; StringType = std::__cxx11::basic_string<char>]’: /usr/include/gtest/gtest-matchers.h:484:47: required from ‘bool testing::PolymorphicMatcher<Impl>::MonomorphicImpl<T>::MatchAndExplain(T, testing::MatchResultListener*) const [with T = const std::basic_string_view<char>&; Impl = testing::internal::HasSubstrMatcher<std::__cxx11::basic_string<char> >]’ /usr/include/gtest/gtest-matchers.h:483:18: required from here /usr/include/gmock/gmock-matchers.h:760:23: error: invalid initialization of reference of type ‘const std::__cxx11::basic_string<char>&’ from expression of type ‘const std::basic_string_view<char>’ 760 | const StringType& s2(s); ``` after, works --- Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/facebook/sapling/pull/692). * facebook/sapling#693 * facebook/sapling#696 * __->__ facebook/sapling#692 * facebook/sapling#691 * facebook/sapling#682 * facebook/sapling#689 * facebook/sapling#697 * facebook/sapling#706 * facebook/sapling#730 Reviewed By: xavierd Differential Revision: D49874525 Pulled By: genevievehelsel fbshipit-source-id: f13c7cfa6b92eb0775f8d39181b4a4573459e777
facebook-github-bot
pushed a commit
to facebookincubator/velox
that referenced
this pull request
Oct 7, 2023
Summary: Pull Request resolved: #6925 save some CI diskspace, use ubuntu 20.04 and later libgit2 Save some time and diskspace in CI builds for github by using the system libraries X-link: facebook/sapling#693 Test Plan: Install deps and build locally with ``` sudo ./build/fbcode_builder/getdeps.py --allow-system-packages install-system-deps --recursive eden ... ./build/fbcode_builder/getdeps.py --allow-system-packages build --src-dir=. eden ``` before: we build libgit2 from source after: we use the system library --- Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/facebook/sapling/pull/693). * __->__ facebook/sapling#693 * facebook/sapling#696 * facebook/sapling#692 * facebook/sapling#691 * facebook/sapling#682 * facebook/sapling#689 * facebook/sapling#697 * facebook/sapling#706 * facebook/sapling#730 Reviewed By: quark-zju, xavierd Differential Revision: D49874384 Pulled By: genevievehelsel fbshipit-source-id: 09c01eec69446c9bfa95861d9e71617b44b44fce
facebook-github-bot
pushed a commit
to facebookexperimental/rust-shed
that referenced
this pull request
Oct 7, 2023
Summary: X-link: facebookincubator/velox#6925 save some CI diskspace, use ubuntu 20.04 and later libgit2 Save some time and diskspace in CI builds for github by using the system libraries X-link: facebook/sapling#693 Reviewed By: quark-zju, xavierd Differential Revision: D49874384 Pulled By: genevievehelsel fbshipit-source-id: 09c01eec69446c9bfa95861d9e71617b44b44fce
facebook-github-bot
pushed a commit
to facebookexperimental/edencommon
that referenced
this pull request
Oct 7, 2023
Summary: X-link: facebookincubator/velox#6925 save some CI diskspace, use ubuntu 20.04 and later libgit2 Save some time and diskspace in CI builds for github by using the system libraries X-link: facebook/sapling#693 Reviewed By: quark-zju, xavierd Differential Revision: D49874384 Pulled By: genevievehelsel fbshipit-source-id: 09c01eec69446c9bfa95861d9e71617b44b44fce
facebook-github-bot
pushed a commit
to facebook/fboss
that referenced
this pull request
Oct 7, 2023
Summary: X-link: facebookincubator/velox#6925 save some CI diskspace, use ubuntu 20.04 and later libgit2 Save some time and diskspace in CI builds for github by using the system libraries X-link: facebook/sapling#693 Reviewed By: quark-zju, xavierd Differential Revision: D49874384 Pulled By: genevievehelsel fbshipit-source-id: 09c01eec69446c9bfa95861d9e71617b44b44fce
facebook-github-bot
pushed a commit
to facebook/fb303
that referenced
this pull request
Oct 7, 2023
Summary: X-link: facebookincubator/velox#6925 save some CI diskspace, use ubuntu 20.04 and later libgit2 Save some time and diskspace in CI builds for github by using the system libraries X-link: facebook/sapling#693 Reviewed By: quark-zju, xavierd Differential Revision: D49874384 Pulled By: genevievehelsel fbshipit-source-id: 09c01eec69446c9bfa95861d9e71617b44b44fce
facebook-github-bot
pushed a commit
to facebook/folly
that referenced
this pull request
Oct 7, 2023
Summary: X-link: facebookincubator/velox#6925 save some CI diskspace, use ubuntu 20.04 and later libgit2 Save some time and diskspace in CI builds for github by using the system libraries X-link: facebook/sapling#693 Reviewed By: quark-zju, xavierd Differential Revision: D49874384 Pulled By: genevievehelsel fbshipit-source-id: 09c01eec69446c9bfa95861d9e71617b44b44fce
facebook-github-bot
pushed a commit
to facebook/watchman
that referenced
this pull request
Oct 7, 2023
Summary: X-link: facebookincubator/velox#6925 save some CI diskspace, use ubuntu 20.04 and later libgit2 Save some time and diskspace in CI builds for github by using the system libraries X-link: facebook/sapling#693 Reviewed By: quark-zju, xavierd Differential Revision: D49874384 Pulled By: genevievehelsel fbshipit-source-id: 09c01eec69446c9bfa95861d9e71617b44b44fce
facebook-github-bot
pushed a commit
that referenced
this pull request
Oct 7, 2023
Summary: X-link: facebookincubator/velox#6923 fix mononoke linux local cargo build with lld During my recent reviewstack changes I noticed that mononoke links fine on macOS, but github PRs are failing on linux with duplicate zstd symbols during linking. To fix this, switch to lld for the final link on linux by setting the cargo options, which needs a getdeps cargo.py change to ensure they aren't overwritten It works locally but looks like its running our of disk space on github. Next change in stack addresses github CI diskspace Pull Request resolved: #697 Test Plan: Install the new lld system dep with `sudo ./build/fbcode_builder/getdeps.py install-system-deps --recursive mononoke` Run a build locally with `./build/fbcode_builder/getdeps.py --allow-system-packages build --src-dir=. mononoke` Before, zstd link errors like https://github.com/facebook/sapling/actions/runs/5432912652/jobs/9880272333: ``` /usr/bin/ld: /home/alex/local/tmp/toolbox/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/build/mononoke/debug/deps/libzstd_sys-837ebc89eb1d31b6.rlib(zstd_lazy.o): in function `ZSTD_compressBlock_btlazy2_extDict': zstd_lazy.c:(.text.ZSTD_compressBlock_btlazy2_extDict+0x0): multiple definition of `ZSTD_compressBlock_btlazy2_extDict'; /home/alex/local/tmp/toolbox/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/build/mononoke/debug/deps/libzstd_legacy_mononoke_sys-78a66a56bd363eb2.rlib(zstd_lazy.o):zstd_lazy.c:(.text.ZSTD_compressBlock_btlazy2_extDict+0x0): first defined here collect2: error: ld returned 1 exit status ``` After, works: ``` Compiling mononoke v0.1.0 (/home/alex/local/tmp/toolbox/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/build/mononoke/source/eden/mononoke/server) Finished dev [unoptimized] target(s) in 1m 17s ``` --- Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/facebook/sapling/pull/697). * #693 * #696 * #692 * #691 * #682 * #689 * __->__ #697 * #706 * #730 Reviewed By: sggutier Differential Revision: D49875277 Pulled By: genevievehelsel fbshipit-source-id: 998f525fd71333b31747d0ad0e0dda3ebe2b0796
facebook-github-bot
pushed a commit
that referenced
this pull request
Oct 7, 2023
Summary: X-link: facebookincubator/velox#6927 allow getdeps github actions to free up disk Allow getdeps to free up some disk from the runner and intermediate build steps as some runs (notably the linux eden and mononoke ones) are hitting disk space limits Pull Request resolved: #689 Test Plan: Regenerate and run CI for mononoke. Mononoke Mac should continue to work Mononoke Linux should go from broken to working ``` ./build/fbcode_builder/getdeps.py --allow-system-packages generate-github-actions --free-up-disk --src-dir=. --output-dir=.github/workflows --job-name="Mononoke " --job-file-prefix=mononoke_ mononoke ``` --- Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/facebook/sapling/pull/689). * #693 * #696 * #692 * #691 * #682 * __->__ #689 * #697 * #706 * #730 Reviewed By: sggutier Differential Revision: D49875256 Pulled By: genevievehelsel fbshipit-source-id: b85b6b2f11857670915b64f47d3c0abd4ca8ca31
facebook-github-bot
pushed a commit
that referenced
this pull request
Oct 7, 2023
Summary: X-link: facebookincubator/velox#6924 fix getdeps actions generation for rust toolchain Generated actions files created from getdeps.py were missing rust toolchain which meant annoying manual edits This change also makes the github actions generation honor the --no-tests argument, which is useful when regenerating the edenfs actions Pull Request resolved: #682 Test Plan: To test we add the missing undeclared rust-shed dependency to the eden manifest and regenerate. Generate updated actions yaml with: ``` ./build/fbcode_builder/getdeps.py --allow-system-packages generate-github-actions --no-tests --free-up-disk --os-type=linux --src-dir=. --output-dir=.github/workflows --job-name "EdenFS " --job-file-prefix=edenfs_ eden ``` Before, was missing the rust toolchain in the auto regenerated eden actions file. After, toolchain is present Before, was missing the --no-tests in the "Build eden" step of the auto-regenerated eden actions file. After, has it Eden stil needs some other fixes to build, which follow in this stqck --- Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/facebook/sapling/pull/682). * #693 * #696 * #692 * #691 * __->__ #682 * #689 * #697 * #706 * #730 Reviewed By: sggutier Differential Revision: D49875258 Pulled By: genevievehelsel fbshipit-source-id: 173f86083ba92bab4063813d3e392df428b9ffe4
facebook-github-bot
pushed a commit
that referenced
this pull request
Oct 7, 2023
Summary: edenfs: add missing cmake edencommon dependency edenfs was missing a declared dep on edencommon in the rpc code CMakeLists.txt, this fixed missing edencommon headers Pull Request resolved: #691 Test Plan: Build locally: `./build/fbcode_builder/getdeps.py --allow-system-packages build --src-dir=. eden` Before: ``` FAILED: eden/fs/nfs/portmap/CMakeFiles/eden_nfs_portmap_server.dir/Rpcbindd.cpp.o /usr/bin/c++ -DBOOST_ALL_NO_LIB -DFOLLY_XLOG_STRIP_PREFIXES=\"/home/alex/local/sapling:/home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/build/eden\" -DGFLAGS_IS_A_DLL=0 -I. -I/home/alex/local/sapling -I/home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/fb303/include -I/home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/fizz/include -I/home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/wangle/include -I/home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/fbthrift/include -I/home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/blake3-lrolsLBeklWWBLYfMM4fF2TJGwCeReSgHBUbEgA3TFw/include -isystem /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/folly/include -isystem /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/fmt-_ZorQJC2eeu9K7XicUjMKUe54C8sh6ZsaDZIZfQ_QmE/include -g -Wall -Wextra -Wno-deprecated -Wno-deprecated-declarations -Wno-nullability-completeness -Wno-unknown-pragmas -O2 -g -DNDEBUG -pthread -std=gnu++17 -MD -MT eden/fs/nfs/portmap/CMakeFiles/eden_nfs_portmap_server.dir/Rpcbindd.cpp.o -MF eden/fs/nfs/portmap/CMakeFiles/eden_nfs_portmap_server.dir/Rpcbindd.cpp.o.d -o eden/fs/nfs/portmap/CMakeFiles/eden_nfs_portmap_server.dir/Rpcbindd.cpp.o -c /home/alex/local/sapling/eden/fs/nfs/portmap/Rpcbindd.cpp In file included from /home/alex/local/sapling/eden/fs/nfs/portmap/Rpcbindd.h:14, from /home/alex/local/sapling/eden/fs/nfs/portmap/Rpcbindd.cpp:8: /home/alex/local/sapling/eden/fs/utils/PathFuncs.h:24:10: fatal error: eden/common/utils/StringConv.h: No such file or directory 24 | #include "eden/common/utils/StringConv.h" | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated. ``` After, works and gets to the next error, see next in stack --- Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/facebook/sapling/pull/691). * #693 * #696 * #692 * __->__ #691 * #682 * #689 * #697 * #706 * #730 Reviewed By: xavierd Differential Revision: D49874584 Pulled By: genevievehelsel fbshipit-source-id: cc55efff959f6230bdd3c6dd01d16a02db472a48
facebook-github-bot
pushed a commit
that referenced
this pull request
Oct 7, 2023
Summary: X-link: facebookincubator/velox#6926 fix edenfs oss build, ubuntu googletest is too old Update the googletest manifest as ubuntu's LTS packages are too old. 22.10 on looks fine Pull Request resolved: #692 Test Plan: Build locally with ./build/fbcode_builder/getdeps.py --allow-system-packages build --src-dir=. eden before, breaks at: ``` /usr/bin/c++ -DBOOST_ALL_NO_LIB -DFOLLY_XLOG_STRIP_PREFIXES=\"/home/alex/local/sapling:/home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/build/eden\" -DGFLAGS_IS_A_DLL=0 -I. -I/home/alex/local/sapling -I/home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/blake3-lrolsLBeklWWBLYfMM4fF2TJGwCeReSgHBUbEgA3TFw/include -isystem /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/folly/include -isystem /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/fb303/include -isystem /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/fizz/include -isystem /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/wangle/include -isystem /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/fbthrift/include -isystem /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/fmt-_ZorQJC2eeu9K7XicUjMKUe54C8sh6ZsaDZIZfQ_QmE/include -isystem /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/mvfst/include -isystem /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/edencommon/include -isystem /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/cpptoml-_zeUSHY925tqbka8WnSRFSO5ZQbE56aQYsrCa6wv7mU/include -isystem /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/rocksdb-Y6wy710FNBv6sNEe4kCJ_-a3O5-tys4ivojRziE0GyM/include -g -Wall -Wextra -Wno-deprecated -Wno-deprecated-declarations -Wno-nullability-completeness -Wno-unknown-pragmas -O2 -g -DNDEBUG -fsized-deallocation -pthread -std=gnu++17 -MD -MT eden/fs/utils/test/CMakeFiles/eden_utils_test.dir/SourceLocationTest.cpp.o -MF eden/fs/utils/test/CMakeFiles/eden_utils_test.dir/SourceLocationTest.cpp.o.d -o eden/fs/utils/test/CMakeFiles/eden_utils_test.dir/SourceLocationTest.cpp.o -c /home/alex/local/sapling/eden/fs/utils/test/SourceLocationTest.cpp In file included from /usr/include/gmock/gmock-spec-builders.h:75, from /usr/include/gmock/gmock-generated-function-mockers.h:47, from /usr/include/gmock/gmock-function-mocker.h:39, from /usr/include/gmock/gmock.h:61, from /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/folly/include/folly/portability/GMock.h:32, from /home/alex/local/sapling/eden/fs/utils/test/SourceLocationTest.cpp:11: /usr/include/gmock/gmock-matchers.h: In instantiation of ‘bool testing::internal::HasSubstrMatcher<StringType>::MatchAndExplain(const MatcheeStringType&, testing::MatchResultListener*) const [with MatcheeStringType = std::basic_string_view<char>; StringType = std::__cxx11::basic_string<char>]’: /usr/include/gtest/gtest-matchers.h:484:47: required from ‘bool testing::PolymorphicMatcher<Impl>::MonomorphicImpl<T>::MatchAndExplain(T, testing::MatchResultListener*) const [with T = const std::basic_string_view<char>&; Impl = testing::internal::HasSubstrMatcher<std::__cxx11::basic_string<char> >]’ /usr/include/gtest/gtest-matchers.h:483:18: required from here /usr/include/gmock/gmock-matchers.h:760:23: error: invalid initialization of reference of type ‘const std::__cxx11::basic_string<char>&’ from expression of type ‘const std::basic_string_view<char>’ 760 | const StringType& s2(s); ``` after, works --- Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/facebook/sapling/pull/692). * #693 * #696 * __->__ #692 * #691 * #682 * #689 * #697 * #706 * #730 Reviewed By: xavierd Differential Revision: D49874525 Pulled By: genevievehelsel fbshipit-source-id: f13c7cfa6b92eb0775f8d39181b4a4573459e777
facebook-github-bot
pushed a commit
that referenced
this pull request
Oct 7, 2023
Summary: edenfs fix redundant move warnings in ImmediateFuture-inl.h GCC complained about these redundant moves, spamming the logs. Pull Request resolved: #696 Test Plan: Build locally with ./build/fbcode_builder/getdeps.py --allow-system-packages build --src-dir=. eden before, warnings: ``` /home/runner/work/sapling/sapling/eden/fs/utils/ImmediateFuture-inl.h:356:25: warning: redundant move in return statement [-Wredundant-move] 356 | return std::move(res); | ^ /home/runner/work/sapling/sapling/eden/fs/utils/ImmediateFuture-inl.h:356:25: note: remove ‘std::move’ call /home/runner/work/sapling/sapling/eden/fs/utils/ImmediateFuture-inl.h: In instantiation of ‘facebook::eden::ImmediateFuture<std::vector<folly::Try<T> > > facebook::eden::collectAll(std::vector<facebook::eden::ImmediateFuture<T> >) [with T = {anonymous}::ImmediateFuture_collectUncopyable_Test::TestBody()::Uncopyable]’: /home/runner/work/sapling/sapling/eden/fs/utils/test/ImmediateFutureTest.cpp:350:39: required from here /home/runner/work/sapling/sapling/eden/fs/utils/ImmediateFuture-inl.h:356:25: warning: redundant move in return statement [-Wredundant-move] /home/runner/work/sapling/sapling/eden/fs/utils/ImmediateFuture-inl.h:356:25: note: remove ‘std::move’ call /home/runner/work/sapling/sapling/eden/fs/utils/ImmediateFuture-inl.h: In instantiation of ‘facebook::eden::ImmediateFuture<typename facebook::eden::detail::continuation_result<typename std::enable_if<is_invocable_v<Func, folly::Try<T> >, void>::type, Func, folly::Try<T> >::type> facebook::eden::ImmediateFuture<T>::thenTry(Func&&) && [with Func = facebook::eden::ImmediateFuture<T>::ensure(Func&&) && [with Func = {anonymous}::ImmediateFuture_ensureBasic_Test::TestBody()::<lambda()>&; T = int]::<lambda(folly::Try<int>)>; T = int; typename facebook::eden::detail::continuation_result<typename std::enable_if<is_invocable_v<Func, folly::Try<T> >, void>::type, Func, folly::Try<T> >::type = int]’: /home/runner/work/sapling/sapling/eden/fs/utils/ImmediateFuture-inl.h:196:8: required from ‘facebook::eden::ImmediateFuture<T> facebook::eden::ImmediateFuture<T>::ensure(Func&&) && [with Func = {anonymous}::ImmediateFuture_ensureBasic_Test::TestBody()::<lambda()>&; T = int]’ /home/runner/work/sapling/sapling/eden/fs/utils/test/ImmediateFutureTest.cpp:52:41: required from here /home/runner/work/sapling/sapling/eden/fs/utils/ImmediateFuture-inl.h:240:31: warning: redundant move in return statement [-Wredundant-move] 240 | return std::move(semiFut); ``` after, they are gone! --- Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/facebook/sapling/pull/696). * #693 * __->__ #696 * #692 * #691 * #682 * #689 * #697 * #706 * #730 Reviewed By: xavierd Differential Revision: D49874503 Pulled By: genevievehelsel fbshipit-source-id: 959af45431444b65446eb3f21d1aaf8b4dc5bf30
|
@genevievehelsel merged this pull request in 7cd4cec. |
facebook-github-bot
pushed a commit
to facebookincubator/hsthrift
that referenced
this pull request
Oct 11, 2023
Summary: X-link: facebookincubator/velox#6925 save some CI diskspace, use ubuntu 20.04 and later libgit2 Save some time and diskspace in CI builds for github by using the system libraries X-link: facebook/sapling#693 Reviewed By: quark-zju, xavierd Differential Revision: D49874384 Pulled By: genevievehelsel fbshipit-source-id: 09c01eec69446c9bfa95861d9e71617b44b44fce
ericyuliu
pushed a commit
to ericyuliu/velox
that referenced
this pull request
Oct 12, 2023
Summary: Pull Request resolved: facebookincubator#6923 fix mononoke linux local cargo build with lld During my recent reviewstack changes I noticed that mononoke links fine on macOS, but github PRs are failing on linux with duplicate zstd symbols during linking. To fix this, switch to lld for the final link on linux by setting the cargo options, which needs a getdeps cargo.py change to ensure they aren't overwritten It works locally but looks like its running our of disk space on github. Next change in stack addresses github CI diskspace X-link: facebook/sapling#697 Test Plan: Install the new lld system dep with `sudo ./build/fbcode_builder/getdeps.py install-system-deps --recursive mononoke` Run a build locally with `./build/fbcode_builder/getdeps.py --allow-system-packages build --src-dir=. mononoke` Before, zstd link errors like https://github.com/facebook/sapling/actions/runs/5432912652/jobs/9880272333: ``` /usr/bin/ld: /home/alex/local/tmp/toolbox/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/build/mononoke/debug/deps/libzstd_sys-837ebc89eb1d31b6.rlib(zstd_lazy.o): in function `ZSTD_compressBlock_btlazy2_extDict': zstd_lazy.c:(.text.ZSTD_compressBlock_btlazy2_extDict+0x0): multiple definition of `ZSTD_compressBlock_btlazy2_extDict'; /home/alex/local/tmp/toolbox/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/build/mononoke/debug/deps/libzstd_legacy_mononoke_sys-78a66a56bd363eb2.rlib(zstd_lazy.o):zstd_lazy.c:(.text.ZSTD_compressBlock_btlazy2_extDict+0x0): first defined here collect2: error: ld returned 1 exit status ``` After, works: ``` Compiling mononoke v0.1.0 (/home/alex/local/tmp/toolbox/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/build/mononoke/source/eden/mononoke/server) Finished dev [unoptimized] target(s) in 1m 17s ``` --- Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/facebook/sapling/pull/697). * facebook/sapling#693 * facebook/sapling#696 * facebook/sapling#692 * facebook/sapling#691 * facebook/sapling#682 * facebook/sapling#689 * __->__ facebook/sapling#697 * facebook/sapling#706 * facebook/sapling#730 Reviewed By: sggutier Differential Revision: D49875277 Pulled By: genevievehelsel fbshipit-source-id: 998f525fd71333b31747d0ad0e0dda3ebe2b0796
ericyuliu
pushed a commit
to ericyuliu/velox
that referenced
this pull request
Oct 12, 2023
Summary: Pull Request resolved: facebookincubator#6927 allow getdeps github actions to free up disk Allow getdeps to free up some disk from the runner and intermediate build steps as some runs (notably the linux eden and mononoke ones) are hitting disk space limits X-link: facebook/sapling#689 Test Plan: Regenerate and run CI for mononoke. Mononoke Mac should continue to work Mononoke Linux should go from broken to working ``` ./build/fbcode_builder/getdeps.py --allow-system-packages generate-github-actions --free-up-disk --src-dir=. --output-dir=.github/workflows --job-name="Mononoke " --job-file-prefix=mononoke_ mononoke ``` --- Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/facebook/sapling/pull/689). * facebook/sapling#693 * facebook/sapling#696 * facebook/sapling#692 * facebook/sapling#691 * facebook/sapling#682 * __->__ facebook/sapling#689 * facebook/sapling#697 * facebook/sapling#706 * facebook/sapling#730 Reviewed By: sggutier Differential Revision: D49875256 Pulled By: genevievehelsel fbshipit-source-id: b85b6b2f11857670915b64f47d3c0abd4ca8ca31
ericyuliu
pushed a commit
to ericyuliu/velox
that referenced
this pull request
Oct 12, 2023
…6924) Summary: Pull Request resolved: facebookincubator#6924 fix getdeps actions generation for rust toolchain Generated actions files created from getdeps.py were missing rust toolchain which meant annoying manual edits This change also makes the github actions generation honor the --no-tests argument, which is useful when regenerating the edenfs actions X-link: facebook/sapling#682 Test Plan: To test we add the missing undeclared rust-shed dependency to the eden manifest and regenerate. Generate updated actions yaml with: ``` ./build/fbcode_builder/getdeps.py --allow-system-packages generate-github-actions --no-tests --free-up-disk --os-type=linux --src-dir=. --output-dir=.github/workflows --job-name "EdenFS " --job-file-prefix=edenfs_ eden ``` Before, was missing the rust toolchain in the auto regenerated eden actions file. After, toolchain is present Before, was missing the --no-tests in the "Build eden" step of the auto-regenerated eden actions file. After, has it Eden stil needs some other fixes to build, which follow in this stqck --- Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/facebook/sapling/pull/682). * facebook/sapling#693 * facebook/sapling#696 * facebook/sapling#692 * facebook/sapling#691 * __->__ facebook/sapling#682 * facebook/sapling#689 * facebook/sapling#697 * facebook/sapling#706 * facebook/sapling#730 Reviewed By: sggutier Differential Revision: D49875258 Pulled By: genevievehelsel fbshipit-source-id: 173f86083ba92bab4063813d3e392df428b9ffe4
ericyuliu
pushed a commit
to ericyuliu/velox
that referenced
this pull request
Oct 12, 2023
…#6926) Summary: Pull Request resolved: facebookincubator#6926 fix edenfs oss build, ubuntu googletest is too old Update the googletest manifest as ubuntu's LTS packages are too old. 22.10 on looks fine X-link: facebook/sapling#692 Test Plan: Build locally with ./build/fbcode_builder/getdeps.py --allow-system-packages build --src-dir=. eden before, breaks at: ``` /usr/bin/c++ -DBOOST_ALL_NO_LIB -DFOLLY_XLOG_STRIP_PREFIXES=\"/home/alex/local/sapling:/home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/build/eden\" -DGFLAGS_IS_A_DLL=0 -I. -I/home/alex/local/sapling -I/home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/blake3-lrolsLBeklWWBLYfMM4fF2TJGwCeReSgHBUbEgA3TFw/include -isystem /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/folly/include -isystem /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/fb303/include -isystem /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/fizz/include -isystem /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/wangle/include -isystem /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/fbthrift/include -isystem /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/fmt-_ZorQJC2eeu9K7XicUjMKUe54C8sh6ZsaDZIZfQ_QmE/include -isystem /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/mvfst/include -isystem /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/edencommon/include -isystem /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/cpptoml-_zeUSHY925tqbka8WnSRFSO5ZQbE56aQYsrCa6wv7mU/include -isystem /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/rocksdb-Y6wy710FNBv6sNEe4kCJ_-a3O5-tys4ivojRziE0GyM/include -g -Wall -Wextra -Wno-deprecated -Wno-deprecated-declarations -Wno-nullability-completeness -Wno-unknown-pragmas -O2 -g -DNDEBUG -fsized-deallocation -pthread -std=gnu++17 -MD -MT eden/fs/utils/test/CMakeFiles/eden_utils_test.dir/SourceLocationTest.cpp.o -MF eden/fs/utils/test/CMakeFiles/eden_utils_test.dir/SourceLocationTest.cpp.o.d -o eden/fs/utils/test/CMakeFiles/eden_utils_test.dir/SourceLocationTest.cpp.o -c /home/alex/local/sapling/eden/fs/utils/test/SourceLocationTest.cpp In file included from /usr/include/gmock/gmock-spec-builders.h:75, from /usr/include/gmock/gmock-generated-function-mockers.h:47, from /usr/include/gmock/gmock-function-mocker.h:39, from /usr/include/gmock/gmock.h:61, from /home/alex/local/tmp/ubuntu-20.04/fbcode_builder_getdeps-ZhomeZalexZlocalZsaplingZbuildZfbcode_builder/installed/folly/include/folly/portability/GMock.h:32, from /home/alex/local/sapling/eden/fs/utils/test/SourceLocationTest.cpp:11: /usr/include/gmock/gmock-matchers.h: In instantiation of ‘bool testing::internal::HasSubstrMatcher<StringType>::MatchAndExplain(const MatcheeStringType&, testing::MatchResultListener*) const [with MatcheeStringType = std::basic_string_view<char>; StringType = std::__cxx11::basic_string<char>]’: /usr/include/gtest/gtest-matchers.h:484:47: required from ‘bool testing::PolymorphicMatcher<Impl>::MonomorphicImpl<T>::MatchAndExplain(T, testing::MatchResultListener*) const [with T = const std::basic_string_view<char>&; Impl = testing::internal::HasSubstrMatcher<std::__cxx11::basic_string<char> >]’ /usr/include/gtest/gtest-matchers.h:483:18: required from here /usr/include/gmock/gmock-matchers.h:760:23: error: invalid initialization of reference of type ‘const std::__cxx11::basic_string<char>&’ from expression of type ‘const std::basic_string_view<char>’ 760 | const StringType& s2(s); ``` after, works --- Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/facebook/sapling/pull/692). * facebook/sapling#693 * facebook/sapling#696 * __->__ facebook/sapling#692 * facebook/sapling#691 * facebook/sapling#682 * facebook/sapling#689 * facebook/sapling#697 * facebook/sapling#706 * facebook/sapling#730 Reviewed By: xavierd Differential Revision: D49874525 Pulled By: genevievehelsel fbshipit-source-id: f13c7cfa6b92eb0775f8d39181b4a4573459e777
ericyuliu
pushed a commit
to ericyuliu/velox
that referenced
this pull request
Oct 12, 2023
…ncubator#6925) Summary: Pull Request resolved: facebookincubator#6925 save some CI diskspace, use ubuntu 20.04 and later libgit2 Save some time and diskspace in CI builds for github by using the system libraries X-link: facebook/sapling#693 Test Plan: Install deps and build locally with ``` sudo ./build/fbcode_builder/getdeps.py --allow-system-packages install-system-deps --recursive eden ... ./build/fbcode_builder/getdeps.py --allow-system-packages build --src-dir=. eden ``` before: we build libgit2 from source after: we use the system library --- Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/facebook/sapling/pull/693). * __->__ facebook/sapling#693 * facebook/sapling#696 * facebook/sapling#692 * facebook/sapling#691 * facebook/sapling#682 * facebook/sapling#689 * facebook/sapling#697 * facebook/sapling#706 * facebook/sapling#730 Reviewed By: quark-zju, xavierd Differential Revision: D49874384 Pulled By: genevievehelsel fbshipit-source-id: 09c01eec69446c9bfa95861d9e71617b44b44fce
facebook-github-bot
pushed a commit
to facebook/hhvm
that referenced
this pull request
Oct 23, 2023
Summary: X-link: facebookincubator/velox#6925 save some CI diskspace, use ubuntu 20.04 and later libgit2 Save some time and diskspace in CI builds for github by using the system libraries X-link: facebook/sapling#693 Reviewed By: quark-zju, xavierd Differential Revision: D49874384 Pulled By: genevievehelsel fbshipit-source-id: 09c01eec69446c9bfa95861d9e71617b44b44fce
facebook-github-bot
pushed a commit
to facebook/openr
that referenced
this pull request
Jan 9, 2024
Summary: X-link: facebookincubator/velox#6925 save some CI diskspace, use ubuntu 20.04 and later libgit2 Save some time and diskspace in CI builds for github by using the system libraries X-link: facebook/sapling#693 Reviewed By: quark-zju, xavierd Differential Revision: D49874384 Pulled By: genevievehelsel fbshipit-source-id: 09c01eec69446c9bfa95861d9e71617b44b44fce
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
save some CI diskspace, use ubuntu 20.04 and later libgit2
Save some time and diskspace in CI builds for github by using the system libraries
Test plan:
Install deps and build locally with
before: we build libgit2 from source
after: we use the system library
Stack created with Sapling. Best reviewed with ReviewStack.