Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't treat git repos as non-existent when ignore_git is set #87443

Merged
merged 1 commit into from
Jul 28, 2021

Conversation

jyn514
Copy link
Member

@jyn514 jyn514 commented Jul 24, 2021

The new submodule handling depends on is_git() to be accurate to
decide whether it should handle submodules at all or not. Unfortunately,
is_git() treated "this directory does not have a git repository" and
"this repository should not be used for SHA/version/commit date info"
the same. This changes it to distinguish the two.

To clarify: ignore_get is set by default whenever channel == "dev", which it is by default whenever you're compiling locally. So basically everyone would hit this, not just people who had explicitly configured ignore_git.

Here's an example of an error this fixes:

$ x build
Updating only changed submodules
Submodules updated in 0.01 seconds
    Finished dev [unoptimized + debuginfo] target(s) in 0.17s
warning: x.py has made several changes recently you may want to look at
help: consider looking at the changes in `src/bootstrap/CHANGELOG.md`
note: to silence this warning, add `changelog-seen = 2` at the top of `config.toml`
Building stage0 std artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
    Finished release [optimized] target(s) in 0.16s
Copying stage0 std from stage0 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)
Building LLVM for x86_64-unknown-linux-gnu
detected home dir change, cleaning out entire build directory
running: "cmake" "/home/joshua/rustc3/src/llvm-project/llvm" "-G" "Ninja" "-DLLVM_ENABLE_ASSERTIONS=OFF" "-DLLVM_TARGETS_TO_BUILD=AArch64;ARM;BPF;Hexagon;MSP430;Mips;NVPTX;PowerPC;RISCV;Sparc;SystemZ;WebAssembly;X86" "-DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD=AVR" "-DLLVM_INCLUDE_EXAMPLES=OFF" "-DLLVM_INCLUDE_DOCS=OFF" "-DLLVM_INCLUDE_BENCHMARKS=OFF" "-DLLVM_ENABLE_TERMINFO=OFF" "-DLLVM_ENABLE_LIBEDIT=OFF" "-DLLVM_ENABLE_BINDINGS=OFF" "-DLLVM_ENABLE_Z3_SOLVER=OFF" "-DLLVM_PARALLEL_COMPILE_JOBS=48" "-DLLVM_TARGET_ARCH=x86_64" "-DLLVM_DEFAULT_TARGET_TRIPLE=x86_64-unknown-linux-gnu" "-DLLVM_ENABLE_ZLIB=ON" "-DLLVM_ENABLE_LIBXML2=OFF" "-DLLVM_VERSION_SUFFIX=-rust-dev" "-DCMAKE_INSTALL_MESSAGE=LAZY" "-DCMAKE_C_COMPILER=gcc" "-DCMAKE_CXX_COMPILER=g++" "-DCMAKE_ASM_COMPILER=gcc" "-DCMAKE_C_FLAGS=-ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_CXX_FLAGS=-ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_INSTALL_PREFIX=/home/joshua/rustc3/build/x86_64-unknown-linux-gnu/llvm" "-DCMAKE_ASM_FLAGS= -ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_BUILD_TYPE=Release"
CMake Error: The source directory "/home/joshua/rustc3/src/llvm-project/llvm" does not exist.
Specify --help for usage, or press the help button on the CMake GUI.
thread 'main' panicked at '
command did not execute successfully, got: exit status: 1

build script failed, must exit now', /home/joshua/.local/lib/cargo/registry/src/github.com-1ecc6299db9ec823/cmake-0.1.44/src/lib.rs:885:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
	finished in 0.783 seconds
Build completed unsuccessfully in 0:00:01

I believe this regression was only introduced in #87380, not #82653. @petrochenkov can you check that this fixes the issue you encountered in #82653 (comment) ?

r? @Mark-Simulacrum

@jyn514 jyn514 added the T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) label Jul 24, 2021
@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jul 24, 2021
@jyn514

This comment has been minimized.

@jyn514

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

pub enum GitInfo {
Ignore,
Missing,
Present(Info),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add comments to these variants explaining their meaning? (In particular, Ignore not indicating no information, but rather positive presence, is not obvious).

Alternatively restructuring to Missing and a Present(Option<Info>) variant might be better; it would reduce the explaining needed I think.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add comments to these variants explaining their meaning? (In particular, Ignore not indicating no information, but rather positive presence, is not obvious).

Sure, done.

Alternatively restructuring to Missing and a Present(Option) variant might be better; it would reduce the explaining needed I think.

Hmm, this doesn't seem any more clear to me - you'd still need a comment explaining what Present(None) means.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ignore doesn't make sense in the context of "what is the git info" - it's an outside parameter. Present(None) does need a comment, I agree, but is still imo clearer in terms of present (but no info). Mainly because if I see a match in (future) code for Present, I won't likely think to check for Absent, but this way the check will be more obviously necessary.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Makes sense, I changed it.

@rust-log-analyzer

This comment has been minimized.

The new submodule handling depends on `is_git()` to be accurate to
decide whether it should handle submodules at all or not. Unfortunately,
`is_git()` treated "this directory does not have a git repository" and
"this repository should not be used for SHA/version/commit date info"
the same. This changes it to distinguish the two.

To clarify: ignore_get is set by default whenever channel == "dev", which it is by default whenever you're compiling locally. So basically everyone would hit this, not just people who had explicitly configured ignore_git.

Here's an example of an error this fixes:

```
$ x build
Updating only changed submodules
Submodules updated in 0.01 seconds
    Finished dev [unoptimized + debuginfo] target(s) in 0.17s
warning: x.py has made several changes recently you may want to look at
help: consider looking at the changes in `src/bootstrap/CHANGELOG.md`
note: to silence this warning, add `changelog-seen = 2` at the top of `config.toml`
Building stage0 std artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
    Finished release [optimized] target(s) in 0.16s
Copying stage0 std from stage0 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)
Building LLVM for x86_64-unknown-linux-gnu
detected home dir change, cleaning out entire build directory
running: "cmake" "/home/joshua/rustc3/src/llvm-project/llvm" "-G" "Ninja" "-DLLVM_ENABLE_ASSERTIONS=OFF" "-DLLVM_TARGETS_TO_BUILD=AArch64;ARM;BPF;Hexagon;MSP430;Mips;NVPTX;PowerPC;RISCV;Sparc;SystemZ;WebAssembly;X86" "-DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD=AVR" "-DLLVM_INCLUDE_EXAMPLES=OFF" "-DLLVM_INCLUDE_DOCS=OFF" "-DLLVM_INCLUDE_BENCHMARKS=OFF" "-DLLVM_ENABLE_TERMINFO=OFF" "-DLLVM_ENABLE_LIBEDIT=OFF" "-DLLVM_ENABLE_BINDINGS=OFF" "-DLLVM_ENABLE_Z3_SOLVER=OFF" "-DLLVM_PARALLEL_COMPILE_JOBS=48" "-DLLVM_TARGET_ARCH=x86_64" "-DLLVM_DEFAULT_TARGET_TRIPLE=x86_64-unknown-linux-gnu" "-DLLVM_ENABLE_ZLIB=ON" "-DLLVM_ENABLE_LIBXML2=OFF" "-DLLVM_VERSION_SUFFIX=-rust-dev" "-DCMAKE_INSTALL_MESSAGE=LAZY" "-DCMAKE_C_COMPILER=gcc" "-DCMAKE_CXX_COMPILER=g++" "-DCMAKE_ASM_COMPILER=gcc" "-DCMAKE_C_FLAGS=-ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_CXX_FLAGS=-ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_INSTALL_PREFIX=/home/joshua/rustc3/build/x86_64-unknown-linux-gnu/llvm" "-DCMAKE_ASM_FLAGS= -ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_BUILD_TYPE=Release"
CMake Error: The source directory "/home/joshua/rustc3/src/llvm-project/llvm" does not exist.
Specify --help for usage, or press the help button on the CMake GUI.
thread 'main' panicked at '
command did not execute successfully, got: exit status: 1

build script failed, must exit now', /home/joshua/.local/lib/cargo/registry/src/github.com-1ecc6299db9ec823/cmake-0.1.44/src/lib.rs:885:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
	finished in 0.783 seconds
Build completed unsuccessfully in 0:00:01
```
@jyn514
Copy link
Member Author

jyn514 commented Jul 27, 2021

@bors r+ - it seemed you were ok with the general idea, and I'd like to get this fix in since the behavior is pretty bad without. Happy to make improvements in follow-ups.

@bors
Copy link
Contributor

bors commented Jul 27, 2021

📌 Commit 7cab739 has been approved by jyn514

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jul 27, 2021
JohnTitor added a commit to JohnTitor/rust that referenced this pull request Jul 28, 2021
Don't treat git repos as non-existent when `ignore_git` is set

The new submodule handling depends on `is_git()` to be accurate to
decide whether it should handle submodules at all or not. Unfortunately,
`is_git()` treated "this directory does not have a git repository" and
"this repository should not be used for SHA/version/commit date info"
the same. This changes it to distinguish the two.

To clarify: ignore_get is set by default whenever channel == "dev", which it is by default whenever you're compiling locally. So basically everyone would hit this, not just people who had explicitly configured ignore_git.

Here's an example of an error this fixes:

```
$ x build
Updating only changed submodules
Submodules updated in 0.01 seconds
    Finished dev [unoptimized + debuginfo] target(s) in 0.17s
warning: x.py has made several changes recently you may want to look at
help: consider looking at the changes in `src/bootstrap/CHANGELOG.md`
note: to silence this warning, add `changelog-seen = 2` at the top of `config.toml`
Building stage0 std artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
    Finished release [optimized] target(s) in 0.16s
Copying stage0 std from stage0 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)
Building LLVM for x86_64-unknown-linux-gnu
detected home dir change, cleaning out entire build directory
running: "cmake" "/home/joshua/rustc3/src/llvm-project/llvm" "-G" "Ninja" "-DLLVM_ENABLE_ASSERTIONS=OFF" "-DLLVM_TARGETS_TO_BUILD=AArch64;ARM;BPF;Hexagon;MSP430;Mips;NVPTX;PowerPC;RISCV;Sparc;SystemZ;WebAssembly;X86" "-DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD=AVR" "-DLLVM_INCLUDE_EXAMPLES=OFF" "-DLLVM_INCLUDE_DOCS=OFF" "-DLLVM_INCLUDE_BENCHMARKS=OFF" "-DLLVM_ENABLE_TERMINFO=OFF" "-DLLVM_ENABLE_LIBEDIT=OFF" "-DLLVM_ENABLE_BINDINGS=OFF" "-DLLVM_ENABLE_Z3_SOLVER=OFF" "-DLLVM_PARALLEL_COMPILE_JOBS=48" "-DLLVM_TARGET_ARCH=x86_64" "-DLLVM_DEFAULT_TARGET_TRIPLE=x86_64-unknown-linux-gnu" "-DLLVM_ENABLE_ZLIB=ON" "-DLLVM_ENABLE_LIBXML2=OFF" "-DLLVM_VERSION_SUFFIX=-rust-dev" "-DCMAKE_INSTALL_MESSAGE=LAZY" "-DCMAKE_C_COMPILER=gcc" "-DCMAKE_CXX_COMPILER=g++" "-DCMAKE_ASM_COMPILER=gcc" "-DCMAKE_C_FLAGS=-ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_CXX_FLAGS=-ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_INSTALL_PREFIX=/home/joshua/rustc3/build/x86_64-unknown-linux-gnu/llvm" "-DCMAKE_ASM_FLAGS= -ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_BUILD_TYPE=Release"
CMake Error: The source directory "/home/joshua/rustc3/src/llvm-project/llvm" does not exist.
Specify --help for usage, or press the help button on the CMake GUI.
thread 'main' panicked at '
command did not execute successfully, got: exit status: 1

build script failed, must exit now', /home/joshua/.local/lib/cargo/registry/src/github.com-1ecc6299db9ec823/cmake-0.1.44/src/lib.rs:885:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
	finished in 0.783 seconds
Build completed unsuccessfully in 0:00:01
```

I *believe* this regression was only introduced in rust-lang#87380, not rust-lang#82653. `@petrochenkov` can you check that this fixes the issue you encountered in rust-lang#82653 (comment) ?

r? `@Mark-Simulacrum`
bors added a commit to rust-lang-ci/rust that referenced this pull request Jul 28, 2021
Rollup of 10 pull requests

Successful merges:

 - rust-lang#87315 (Add docs for raw-dylib to unstable book)
 - rust-lang#87330 (Use hashbrown's `extend_reserve()` in `HashMap`)
 - rust-lang#87443 (Don't treat git repos as non-existent when `ignore_git` is set)
 - rust-lang#87453 (Suggest removing unnecessary &mut as help message)
 - rust-lang#87500 (Document math behind MIN/MAX consts on integers)
 - rust-lang#87501 (Remove min_type_alias_impl_trait in favor of type_alias_impl_trait)
 - rust-lang#87507 (SGX mutex is *not* moveable)
 - rust-lang#87513 (bootstrap.py: change `git log` option to indicate desired behavior)
 - rust-lang#87523 (Stop creating a reference then immediately dereferencing it.)
 - rust-lang#87524 (Fix ICE in `diagnostic_hir_wf_check`)

Failed merges:

r? `@ghost`
`@rustbot` modify labels: rollup
@bors bors merged commit 287a252 into rust-lang:master Jul 28, 2021
@rustbot rustbot added this to the 1.56.0 milestone Jul 28, 2021
@jyn514 jyn514 deleted the submodules-take-n branch July 28, 2021 19:27
@petrochenkov
Copy link
Contributor

@jyn514
Looks like it did fix the issue I've been complaining about.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants