pass `CODEGEN_BACKENDS` to docker #117328

lqd · 2023-10-28T19:08:08Z

The backends to build are now defined in the CODEGEN_BACKENDS env var. It's correctly set in CI, but wasn't passed to docker, hence cg_clif wasn't actually built in #81746.

r? @Kobzol: I locally tried CODEGEN_BACKENDS="cranelift" DEPLOY=1 src/ci/docker/run.sh dist-x86_64-linux and this change was enough for ci/run.sh to read the env var.

So I'll try as-is and we'll see.

lqd · 2023-10-28T19:08:31Z

@bors try

bors · 2023-10-28T19:09:40Z

⌛ Trying commit 794bf8a with merge 72df36e...

pass `CODEGEN_BACKENDS` to docker The backends to build are now defined in the `CODEGEN_BACKENDS` env var. It's correctly set in CI, but wasn't passed to docker, hence cg_clif wasn't actually built in rust-lang#81746. r? `@Kobzol:` I locally tried `CODEGEN_BACKENDS="cranelift" DEPLOY=1 src/ci/docker/run.sh dist-x86_64-linux` and this change was enough for `ci/run.sh` to read the env var. However, we can also see people doing both `--env DEPLOY` as-is and `--env BASE_COMMIT="$BASE_COMMIT" \` and I wonder why, do you know? So I'll try as-is and we'll see. (Opened as draft for your feedback, and for me to get a try build going and look at the artifacts)

Either generate it with the actual codegen backend dylib or omit it entirely when the cranelift backend is disabled for this build.

lqd · 2023-10-28T19:23:36Z

(The logs show the cg_clif backend as being built this time)

lqd · 2023-10-28T19:44:08Z

It's starting to look good IMHO. I'll mark this "ready to review" if jakub wants to take a look, but I will still look for the PR builder and test the try build component when/if it's available.

lqd · 2023-10-28T19:48:46Z

Looking good on the PR builder as well: https://github.com/rust-lang/rust/actions/runs/6678763598/job/18150159767?pr=117328#step:25:113 compared to the previous PR https://github.com/rust-lang-ci/rust/actions/runs/6677705890/job/18147987099#step:25:200

lqd · 2023-10-28T20:42:29Z

The bot hasn't posted yet but the try build is done, and works. (Note it was done on the first commit only, but the PR builder worked with both commits)

$ find ~/.rustup/toolchains/72df36e74f2c2df78b3e8802b13aa34540bbf7a2 -name "*cranelift*.so"
/home/lqd/.rustup/toolchains/72df36e74f2c2df78b3e8802b13aa34540bbf7a2/lib/rustlib/x86_64-unknown-linux-gnu/codegen-backends/librustc_codegen_cranelift-1.75.0-nightly.so

$ rustup-toolchain-install-master 72df36e74f2c2df78b3e8802b13aa34540bbf7a2 -c rustc-codegen-cranelift && CARGO_PROFILE_DEV_CODEGEN_BACKEND=cranelift cargo +72df36e74f2c2df78b3e8802b13aa34540bbf7a2 run -q
Hello, world!

(also tested via Cargo.toml changes, not just the env var)

Mark-Simulacrum · 2023-10-28T20:47:20Z

r=me

lqd · 2023-10-28T20:47:48Z

@bors r=Mark-Simulacrum

bors · 2023-10-28T20:47:50Z

📌 Commit aefd7ac has been approved by Mark-Simulacrum

It is now in the queue for this repository.

…acrum pass `CODEGEN_BACKENDS` to docker The backends to build are now defined in the `CODEGEN_BACKENDS` env var. It's correctly set in CI, but wasn't passed to docker, hence cg_clif wasn't actually built in rust-lang#81746. r? `@Kobzol:` I locally tried `CODEGEN_BACKENDS="cranelift" DEPLOY=1 src/ci/docker/run.sh dist-x86_64-linux` and this change was enough for `ci/run.sh` to read the env var. So I'll try as-is and we'll see.

…kingjubilee Rollup of 5 pull requests Successful merges: - rust-lang#115968 (Don't use LFS64 symbols on musl) - rust-lang#117043 (add bootstrap flag `--skip-stage0-validation`) - rust-lang#117082 (Fix closure-inherit-target-feature test for SGX platform) - rust-lang#117312 (memcpy assumptions: link to source showing that GCC makes the same assumption) - rust-lang#117328 (pass `CODEGEN_BACKENDS` to docker) r? `@ghost` `@rustbot` modify labels: rollup

workingjubilee · 2023-10-29T11:32:57Z

Failed in #117339

assertion failed: found_backend

@bors rollup=iffy

…acrum pass `CODEGEN_BACKENDS` to docker The backends to build are now defined in the `CODEGEN_BACKENDS` env var. It's correctly set in CI, but wasn't passed to docker, hence cg_clif wasn't actually built in rust-lang#81746. r? `@Kobzol:` I locally tried `CODEGEN_BACKENDS="cranelift" DEPLOY=1 src/ci/docker/run.sh dist-x86_64-linux` and this change was enough for `ci/run.sh` to read the env var. So I'll try as-is and we'll see.

workingjubilee · 2023-10-29T11:33:21Z

@bors r-

rust-log-analyzer · 2023-10-30T00:04:49Z

The job aarch64-gnu failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

[RUSTC-TIMING] serde_json test:false 1.817
   Compiling tidy v0.1.0 (/checkout/src/tools/tidy)
##[group]Clock drift check

##[error]The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.
  network time: Mon, 30 Oct 2023 00:01:53 GMT
##[endgroup]
##[endgroup]
Session terminated, killing shell...[RUSTC-TIMING] tidy test:false 4.044
 ...killed.
##[error]The operation was canceled.

lqd · 2023-10-30T00:09:49Z

@bors retry The runner has received a shutdown signal

bors · 2023-10-30T01:57:01Z

⌛ Testing commit f656acc with merge 1dfb6b1...

bors · 2023-10-30T03:56:01Z

☀️ Test successful - checks-actions
Approved by: Mark-Simulacrum
Pushing 1dfb6b1 to master...

rust-timer · 2023-10-30T05:36:13Z

Finished benchmarking commit (1dfb6b1): comparison URL.

Overall result: no relevant changes - no action needed

@rustbot label: -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	2.3%	[2.3%, 2.3%]	1
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	2.3%	[2.3%, 2.3%]	1

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	3.0%	[3.0%, 3.0%]	1
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-	-	0

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 635.477s -> 635.247s (-0.04%)
Artifact size: 304.44 MiB -> 304.45 MiB (0.00%)

…-Simulacrum" This reverts commit 1dfb6b1, reversing changes made to bcb5798. That commit broke generating nightly rustc docs. Revert it until we can figure out how to have both, cranelift and docs.

Revert "Auto merge of rust-lang#117328 - lqd:cranelift-rocket, r=Mark-Simulacrum" This reverts commit 1dfb6b1, reversing changes made to bcb5798. That commit broke generating nightly rustc docs. Revert it until we can figure out how to have both, cranelift and docs. Fixes rust-lang#117430

RalfJung · 2023-11-03T09:21:12Z

Unfortunately, this caused #117430 (probably surfacing some pre-existing issue in how docs generation interacts with codegen backend builds), so it is being reverted in #117535.

Revert "Auto merge of rust-lang#117328 - lqd:cranelift-rocket, r=Mark-Simulacrum" This reverts commit 1dfb6b1, reversing changes made to bcb5798. That commit broke generating nightly rustc docs. Revert it until we can figure out how to have both, cranelift and docs. Fixes rust-lang#117430

…g, r=<try> re-enable cranelift on CI First commit addresses the linking issue with compiler crates. Second one ensures that compiler crates are linked correctly, allowing us to detect these hard-to-catch bugs on CI. The remaining three commits cherry-picked from rust-lang#117328 to re-enable the Cranelift backend on CI. More info: rust-lang#117430 cc `@bjorn3` `@RalfJung`

…g, r=<try> re-enable cranelift on CI First commit addresses the linking issue with compiler crates. Second one ensures that compiler crates are linked correctly, allowing us to detect these hard-to-catch bugs on CI. Following three commits cherry-picked from rust-lang#117328 to re-enable the Cranelift backend on CI. More info: rust-lang#117430 cc `@bjorn3` `@RalfJung`

…g, r=Mark-Simulacrum improve compiler&tool documenting and re-enable cranelift on CI First commit addresses the linking issue with compiler crates. Second one ensures that compiler crates are linked correctly (with later commits we added this check for tools as well), allowing us to detect these hard-to-catch bugs on CI. Following three commits cherry-picked from rust-lang#117328 to re-enable the Cranelift backend on CI. More info: rust-lang#117430 cc `@bjorn3` `@RalfJung`

pass CODEGEN_BACKENDS to docker

794bf8a

rustbot assigned Kobzol Oct 28, 2023

rustbot added A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. labels Oct 28, 2023

Ensure the rustc-codegen-cranelift-preview component is never empty

aefd7ac

Either generate it with the actual codegen backend dylib or omit it entirely when the cranelift backend is disabled for this build.

rustbot added the T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) label Oct 28, 2023

bjorn3 mentioned this pull request Oct 28, 2023

Distribute cg_clif as rustup component on the nightly channel #81746

Merged

lqd marked this pull request as ready for review October 28, 2023 19:44

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Oct 28, 2023

workingjubilee mentioned this pull request Oct 29, 2023

Rollup of 5 pull requests #117339

Closed

GuillaumeGomez mentioned this pull request Oct 29, 2023

Rollup of 6 pull requests #117353

Closed

bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Oct 29, 2023

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Oct 30, 2023

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Oct 30, 2023

bors added the merged-by-bors This PR was explicitly merged by bors. label Oct 30, 2023

bors merged commit 1dfb6b1 into rust-lang:master Oct 30, 2023
12 checks passed

rustbot added this to the 1.75.0 milestone Oct 30, 2023

lqd deleted the cranelift-rocket branch October 30, 2023 07:00

oxalica mentioned this pull request Oct 30, 2023

Add rustc-codegen-cranelift component to rust-overlay for nightly oxalica/rust-overlay#142

Closed

github-actions bot mentioned this pull request Oct 31, 2023

Automated pull from upstream master ferrocene/ferrocene#74

Merged

ChrisDenton mentioned this pull request Oct 31, 2023

CI: Bootstrap no longer documents compiler crates #117430

Closed

onur-ozkan mentioned this pull request Nov 4, 2023

improve compiler&tool documenting and re-enable cranelift on CI #117574

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pass `CODEGEN_BACKENDS` to docker #117328

pass `CODEGEN_BACKENDS` to docker #117328

lqd commented Oct 28, 2023 •

edited

Loading

lqd commented Oct 28, 2023

bors commented Oct 28, 2023

lqd commented Oct 28, 2023

lqd commented Oct 28, 2023

lqd commented Oct 28, 2023 •

edited

Loading

lqd commented Oct 28, 2023 •

edited

Loading

Mark-Simulacrum commented Oct 28, 2023

lqd commented Oct 28, 2023

bors commented Oct 28, 2023

workingjubilee commented Oct 29, 2023

workingjubilee commented Oct 29, 2023

rust-log-analyzer commented Oct 30, 2023

lqd commented Oct 30, 2023

bors commented Oct 30, 2023

bors commented Oct 30, 2023

rust-timer commented Oct 30, 2023

RalfJung commented Nov 3, 2023

pass CODEGEN_BACKENDS to docker #117328

pass CODEGEN_BACKENDS to docker #117328

Conversation

lqd commented Oct 28, 2023 • edited Loading

lqd commented Oct 28, 2023

bors commented Oct 28, 2023

lqd commented Oct 28, 2023

lqd commented Oct 28, 2023

lqd commented Oct 28, 2023 • edited Loading

lqd commented Oct 28, 2023 • edited Loading

Mark-Simulacrum commented Oct 28, 2023

lqd commented Oct 28, 2023

bors commented Oct 28, 2023

workingjubilee commented Oct 29, 2023

workingjubilee commented Oct 29, 2023

rust-log-analyzer commented Oct 30, 2023

lqd commented Oct 30, 2023

bors commented Oct 30, 2023

bors commented Oct 30, 2023

rust-timer commented Oct 30, 2023

Overall result: no relevant changes - no action needed

Instruction count

Max RSS (memory usage)

Cycles

Binary size

RalfJung commented Nov 3, 2023

pass `CODEGEN_BACKENDS` to docker #117328

pass `CODEGEN_BACKENDS` to docker #117328

lqd commented Oct 28, 2023 •

edited

Loading

lqd commented Oct 28, 2023 •

edited

Loading

lqd commented Oct 28, 2023 •

edited

Loading