Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

x.py install fails when run outside of Rust source directory #80494

Closed
tmandry opened this issue Dec 29, 2020 · 8 comments
Closed

x.py install fails when run outside of Rust source directory #80494

tmandry opened this issue Dec 29, 2020 · 8 comments
Assignees
@pietroalbini
Labels
C-bug Category: This is a bug. P-high High priority regression-from-stable-to-beta Performance or correctness regression from stable to beta. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue.
Milestone
1.50.0

Comments

@tmandry
Copy link
Member

tmandry commented Dec 29, 2020 

mkdir -p ~/staging/build
mv ~/rust/config.toml ~/staging/build
cd ~/staging/build
~/rust/x.py install

Fails with:

Install std stage2 (Some(TargetSelection { triple: "aarch64-fuchsia", file: None }))
sh: /usr/local/google/home/tmandry/staging/build/build/tmp/dist/rust-std-nightly-aarch64-fuchsia/install.sh: No such file or directory

And the build/tmp/dist directory does not exist. Here are the contents of ~/staging/build/build:

> tree -L 3 build
build
├── bootstrap
│   ├── CACHEDIR.TAG
│   └── debug
│       ├── bootstrap
│       ├── bootstrap.d
│       ├── build
│       ├── deps
│       ├── examples
│       ├── incremental
│       ├── libbootstrap.d
│       ├── libbootstrap.rlib
│       ├── llvm-config-wrapper
│       ├── llvm-config-wrapper.d
│       ├── rustc
│       ├── rustc.d
│       ├── rustdoc
│       ├── rustdoc.d
│       ├── sccache-plus-cl
│       └── sccache-plus-cl.d
├── cache
│   ├── 2020-11-18
│   │   ├── cargo-beta-x86_64-unknown-linux-gnu.tar.gz
│   │   ├── rustc-beta-x86_64-unknown-linux-gnu.tar.gz
│   │   └── rust-std-beta-x86_64-unknown-linux-gnu.tar.gz
│   └── 2020-11-19
│       └── rustfmt-nightly-x86_64-unknown-linux-gnu.tar.gz
├── dist
│   ├── rust-std-nightly-aarch64-fuchsia.tar.gz
│   └── rust-std-nightly-aarch64-fuchsia.tar.xz
├── tmp
│   ├── empty_dir
│   └── tarball
│       ├── rustc
│       └── rust-std
└── x86_64-unknown-linux-gnu
    ├── lld
    │   ├── bin
    │   ├── build
    │   ├── include
    │   ├── lib
    │   └── lld-finished-building
    ├── llvm
    │   ├── bin
    │   ├── build
    │   ├── include
    │   ├── lib
    │   ├── llvm-finished-building
    │   └── share
    ├── stage0
    │   ├── bin
    │   ├── etc
    │   ├── lib
    │   ├── manifest.in
    │   └── share
    ├── stage0-bootstrap-tools
    │   ├── CACHEDIR.TAG
    │   ├── release
    │   └── x86_64-unknown-linux-gnu
    ├── stage0-rustc
    │   ├── CACHEDIR.TAG
    │   ├── release
    │   └── x86_64-unknown-linux-gnu
    ├── stage0-std
    │   ├── release
    │   └── x86_64-unknown-linux-gnu
    ├── stage0-sysroot
    │   └── lib
    ├── stage0-tools-bin
    │   └── fabricate
    ├── stage1
    │   ├── bin
    │   └── lib
    ├── stage1-rustc
    │   ├── CACHEDIR.TAG
    │   ├── release
    │   └── x86_64-unknown-linux-gnu
    ├── stage1-std
    │   ├── release
    │   └── x86_64-unknown-linux-gnu
    ├── stage2
    │   ├── bin
    │   └── lib
    └── stage2-std
        ├── aarch64-fuchsia
        └── release

When run inside the Rust source dir, everything works.
@tmandry tmandry added T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) C-bug Category: This is a bug. regression-from-stable-to-nightly Performance or correctness regression from stable to nightly. labels Dec 29, 2020
@rustbot rustbot added the I-prioritize Issue: Indicates that prioritization has been requested for this issue. label Dec 29, 2020
@Mark-Simulacrum Mark-Simulacrum added P-high High priority T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. and removed I-prioritize Issue: Indicates that prioritization has been requested for this issue. labels Dec 29, 2020
@tmandry
Copy link
Member Author

tmandry commented Dec 29, 2020

Looking back, this first started failing after #79788.

@Mark-Simulacrum
Copy link
Member

cc @pietroalbini - I imagine this is a simple fix but needs to be tracked down

@pietroalbini pietroalbini self-assigned this Dec 30, 2020
@pietroalbini
Copy link
Member

I'll look tomorrow morning.

@pietroalbini pietroalbini mentioned this issue Dec 30, 2020
Merged
@pietroalbini
Copy link
Member

The fix is #80514.

By the way, this bug was not related to being in the same directory as the source code or not. When you were on the same directory you were using an old copy of the needed files in the build directory.

@tmandry
Copy link
Member Author

tmandry commented Dec 30, 2020

Ah, then thanks for the fix and sorry for the confusion!

Dylan-DPC-zz pushed a commit to Dylan-DPC-zz/rust that referenced this issue Dec 31, 2020
@Dylan-DPC
Rollup merge of rust-lang#80514 - pietroalbini:fix-install, r=Mark-Si…
bd54e4a 
…mulacrum

Fix broken ./x.py install

During my tarball refactorings in rust-lang#79788 I changed the directory layout used by the tarball generation code, and that broke the other parts of rustbuild which hardcoded the paths of those directories. Namely, `./x.py install` relied on the uncompressed copy of the tarball left behind by `fabricate`/`rust-installer`, causing rust-lang#80494.

While the easy fix for rust-lang#80494 would've been to just update the hardcoded paths to match the new structure, that fix would leave us in the same situation if we were to change the directory layout again in the future. Instead I refactored the code to return a `GeneratedTarball` struct as the output of all the dist steps, and I put all the paths the rest of rustbuild needs to care about in its fields. That way, future changes to `src/bootstrap/tarball.rs` will not break other stuff.

This PR is best reviewed commit-by-commit.
r? `@Mark-Simulacrum`
`@rustbot` modify labels: beta-nominated beta-accepted T-release
Dylan-DPC-zz pushed a commit to Dylan-DPC-zz/rust that referenced this issue Dec 31, 2020
@Dylan-DPC
Rollup merge of rust-lang#80514 - pietroalbini:fix-install, r=Mark-Si…
5bffc26 
…mulacrum

Fix broken ./x.py install

During my tarball refactorings in rust-lang#79788 I changed the directory layout used by the tarball generation code, and that broke the other parts of rustbuild which hardcoded the paths of those directories. Namely, `./x.py install` relied on the uncompressed copy of the tarball left behind by `fabricate`/`rust-installer`, causing rust-lang#80494.

While the easy fix for rust-lang#80494 would've been to just update the hardcoded paths to match the new structure, that fix would leave us in the same situation if we were to change the directory layout again in the future. Instead I refactored the code to return a `GeneratedTarball` struct as the output of all the dist steps, and I put all the paths the rest of rustbuild needs to care about in its fields. That way, future changes to `src/bootstrap/tarball.rs` will not break other stuff.

This PR is best reviewed commit-by-commit.
r? `@Mark-Simulacrum`
`@rustbot` modify labels: beta-nominated beta-accepted T-release
@pietroalbini
Copy link
Member

Fixed by #80514

@pietroalbini pietroalbini reopened this Jan 6, 2021
@pietroalbini pietroalbini added regression-from-stable-to-beta Performance or correctness regression from stable to beta. and removed regression-from-stable-to-nightly Performance or correctness regression from stable to nightly. labels Jan 6, 2021
@pietroalbini
Copy link
Member

This actually affected beta too, keeping this open to ensure we backport the fix.

@pietroalbini pietroalbini mentioned this issue Jan 6, 2021
Closed
6 tasks
@Mark-Simulacrum Mark-Simulacrum added this to the 1.50.0 milestone Jan 14, 2021
@pietroalbini
Copy link
Member

Backported to beta in #81151! Closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@pietroalbini pietroalbini

Labels
C-bug Category: This is a bug. P-high High priority regression-from-stable-to-beta Performance or correctness regression from stable to beta. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue.
Projects
None yet
Milestone
1.50.0
Development

No branches or pull requests
4 participants
@tmandry @pietroalbini @Mark-Simulacrum @rustbot