Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rollup of 8 pull requests #128000

Closed
wants to merge 37 commits into from
Closed

Conversation

tgross35
Copy link
Contributor

Successful merges:

Failed merges:

r? @ghost
@rustbot modify labels: rollup

Create a similar rollup

momvart and others added 30 commits July 15, 2024 19:54
Signed-off-by: onur-ozkan <work@onurozkan.dev>
sometimes a safety comment is a prayer.
avoid fuzzy provenance casts after deref.

Co-authored-by: Jonas Böttiger <jonasboettiger@icloud.com>
This is technically "not necessary", as we will "just" segfault instead
if we e.g. arrive inside the handler fn with the null altstack. However,
it seems incorrect to go about this hoping that segfaulting is okay,
seeing as how our purpose here is to mitigate stack overflow problems.

Make sure NEED_ALTSTACK syncs with PAGE_SIZE when we do.

Co-authored-by: Jonas Böttiger <jonasboettiger@icloud.com>
Editorialize on the wisdom of this as we do.
Note that current_guard is probably not unsafe for future work.
Co-authored-by: Jonas Böttiger <jonasboettiger@icloud.com>
Now there are separate functions for running a command without capturing, running while capturing stdout
and running while capturing everything. This should help avoid situations where stdout/stderr is accessed
when it was not captured.
…ut/stderr

If e.g. only stdout is captured, but the caller tries to read stderr, previously
they would get back an empty string. Now the code will explicitly panic when
accessing an uncaptured output stream.
Only the last candidate can possibly have more match pairs, so this can be
separate from the main or-candidate postprocessing loop.
When CI rustc is enabled and rustdoc sources are unchanged, we can use
the precompiled rustdoc from the CI rustc's sysroot. This speeds up
bootstrapping quite a lot by avoiding unnecessary rustdoc compilation.

Signed-off-by: onur-ozkan <work@onurozkan.dev>
…Kobzol

 use precompiled rustdoc with CI rustc

When CI rustc is enabled and rustdoc sources are unchanged, we can use the precompiled rustdoc from the CI rustc's sysroot. This speeds up bootstrapping quite a lot by avoiding unnecessary rustdoc compilation.
Migrate 9 more very similar FFI `run-make` tests to rmake

Part of rust-lang#121876 and the associated [Google Summer of Code project](https://blog.rust-lang.org/2024/05/01/gsoc-2024-selected-projects.html).

For the tracking issue:

- return-non-c-like-enum-from-c
- pass-non-c-like-enum-to-c
- c-static-dylib
- c-static-rlib
- extern-fn-generic
- extern-fn-with-union
- lto-no-link-whole-rlib
- linkage-attr-on-static
- issue-28595
…illot

Add a hook for `should_codegen_locally`

This PR lifts the module-local function `should_codegen_locally` to `TyCtxt` as a hook.
In addition to monomorphization, this function is used for checking the dependency of `compiler_builtins` on other libraries. Moving this function to the hooks also makes overriding it possible for the tools that use the rustc interface.
…=onur-ozkan

Bootstrap command refactoring: make command output API more bulletproof (step 7)

Continuation of rust-lang#127680.

This PR modifies the API of running commands to make it more explicit when a command is expected to produce programmatically handled output. Now if you call just `run`, you cannot access the stdout/stderr by accident, because it will not be returned to the caller.

This API change might be seen as overkill, let me know what do you think. In any case, I'd like to land the second commit, to make it harder to accidentally read stdout/stderr of commands that did not capture output (now you'd get an empty string as a result, but you should probably get a panic instead, if you try to read uncaptured stdout/stderr).

Tracking issue: rust-lang#126819

r? `@onur-ozkan`
…ack-overflow-fn, r=joboet

unix: document unsafety for std `sig{action,altstack}`

I found many surprising elements here while trying to wrap a measly 5 functions with `unsafe`. I would rather not "just" mindlessly wrap this code with `unsafe { }`, so I decided to document it properly.

On Unix, this code covers the "create and setup signal handler" part of the stack overflow code, and serves as the primary safety boundary for the signal handler. It is rarely audited, very gnarly, and worth extra attention. It calls other unsafe functions defined in this module, but "can we correctly map the right memory, or find the right address ranges?" are separate questions, and get increasingly platform-specific. The question here is the more general "are we doing everything in the correct order, and setting up the handler in the correct way?"

As part of this audit, I noticed that we do some peculiar things that we should probably refrain from. However, I avoided making changes that I deemed might have a different final result in Rust programs. I did, however, reorder some events so that the signal handler is installed _after_ we install the alternate stack. We do not run much code between these events, but it is probably best if the timespan between the handler being available and the new stack being installed is 0 nanoseconds.
…or-kmc-solid, r=Amanieu

kmc-solid: `#![forbid(unsafe_op_in_unsafe_fn)]`

The path logic _should_ handle the forbiddance in the itron sources correctly, despite them being an "out-of-line" module.
match lowering: Split `finalize_or_candidate` into more coherent methods

I noticed that `finalize_or_candidate` was responsible for several different postprocessing tasks, making it difficult to understand.

This PR aims to clean up some of the confusion by:
- Extracting `remove_never_subcandidates` from `merge_trivial_subcandidates`
- Extracting `test_remaining_match_pairs_after_or` from `finalize_or_candidate`
- Taking what remains of `finalize_or_candidate`, and inlining it into its caller

---
Reviewing individual commits and ignoring whitespace is recommended.

Most of the large-looking changes are just moving existing code around, mostly unaltered.

r? `@Nadrieril`
…thercote

run_make_support: skip rustfmt for lib.rs

To avoid them getting reordered once rust-lang#125443 goes through.

r? `@nnethercote` (since you were working on this)
@rustbot rustbot added A-run-make Area: port run-make Makefiles to rmake.rs A-testsuite Area: The testsuite used to check the correctness of rustc O-solid Operating System: SOLID O-unix Operating system: Unix-like S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-libs Relevant to the library team, which will review and decide on the PR/issue. rollup A PR which is a rollup labels Jul 20, 2024
@tgross35
Copy link
Contributor Author

@bors r+ rollup=never p=8

@bors
Copy link
Contributor

bors commented Jul 20, 2024

📌 Commit bf972c7 has been approved by tgross35

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jul 20, 2024
@Kobzol
Copy link
Contributor

Kobzol commented Jul 20, 2024

I think that #127463 and #127799 will conflict.

@rust-log-analyzer
Copy link
Collaborator

The job mingw-check-tidy failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)
Getting action download info
Download action repository 'msys2/setup-msys2@v2.22.0' (SHA:cc11e9188b693c2b100158c3322424c4cc1dadea)
Download action repository 'actions/checkout@v4' (SHA:692973e3d937129bcbf40652eb9f2f61becf3332)
Download action repository 'actions/upload-artifact@v4' (SHA:0b2256b8c012f0828dc542b3febcab082c67f72b)
Complete job name: PR - mingw-check-tidy
git config --global core.autocrlf false
shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
---
COPY scripts/sccache.sh /scripts/
RUN sh /scripts/sccache.sh

COPY host-x86_64/mingw-check/reuse-requirements.txt /tmp/
RUN pip3 install --no-deps --no-cache-dir --require-hashes -r /tmp/reuse-requirements.txt \
    && pip3 install virtualenv
COPY host-x86_64/mingw-check/validate-toolstate.sh /scripts/
COPY host-x86_64/mingw-check/validate-error-codes.sh /scripts/

# NOTE: intentionally uses python2 for x.py so we can test it still works.
# NOTE: intentionally uses python2 for x.py so we can test it still works.
# validate-toolstate only runs in our CI, so it's ok for it to only support python3.
ENV SCRIPT TIDY_PRINT_DIFF=1 python2.7 ../x.py test \
           --stage 0 src/tools/tidy tidyselftest --extra-checks=py:lint,cpp:fmt
# This file is autogenerated by pip-compile with Python 3.10
# by the following command:
#
#    pip-compile --allow-unsafe --generate-hashes reuse-requirements.in
---

#12 [5/8] COPY host-x86_64/mingw-check/reuse-requirements.txt /tmp/
#12 DONE 0.0s

#13 [6/8] RUN pip3 install --no-deps --no-cache-dir --require-hashes -r /tmp/reuse-requirements.txt     && pip3 install virtualenv
#13 0.431   Downloading binaryornot-0.4.4-py2.py3-none-any.whl (9.0 kB)
#13 0.443 Collecting boolean-py==4.0
#13 0.447   Downloading boolean.py-4.0-py3-none-any.whl (25 kB)
#13 0.458 Collecting chardet==5.1.0
---
#13 3.818 Building wheels for collected packages: reuse
#13 3.819   Building wheel for reuse (pyproject.toml): started
#13 4.160   Building wheel for reuse (pyproject.toml): finished with status 'done'
#13 4.161   Created wheel for reuse: filename=reuse-1.1.0-cp310-cp310-manylinux_2_35_x86_64.whl size=181117 sha256=f5f58750481f69515c2c0d1d503daf565e2565c370d07fc6aeb95fe3498b4269
#13 4.162   Stored in directory: /tmp/pip-ephem-wheel-cache-rpd9e3us/wheels/c2/3c/b9/1120c2ab4bd82694f7e6f0537dc5b9a085c13e2c69a8d0c76d
#13 4.164 Installing collected packages: boolean-py, binaryornot, setuptools, reuse, python-debian, markupsafe, license-expression, jinja2, chardet
#13 4.188   Attempting uninstall: setuptools
#13 4.188     Found existing installation: setuptools 59.6.0
#13 4.190     Not uninstalling setuptools at /usr/lib/python3/dist-packages, outside environment /usr
#13 4.190     Not uninstalling setuptools at /usr/lib/python3/dist-packages, outside environment /usr
#13 4.190     Can't uninstall 'setuptools'. No files were found to uninstall.
#13 4.900 Successfully installed binaryornot-0.4.4 boolean-py-4.0 chardet-5.1.0 jinja2-3.1.2 license-expression-30.0.0 markupsafe-2.1.1 python-debian-0.1.49 reuse-1.1.0 setuptools-66.0.0
#13 4.901 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
#13 5.454 Collecting virtualenv
#13 5.494   Downloading virtualenv-20.26.3-py3-none-any.whl (5.7 MB)
#13 5.582      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.7/5.7 MB 66.2 MB/s eta 0:00:00
#13 5.635 Collecting platformdirs<5,>=3.9.1
#13 5.639   Downloading platformdirs-4.2.2-py3-none-any.whl (18 kB)
#13 5.671 Collecting filelock<4,>=3.12.2
#13 5.674   Downloading filelock-3.15.4-py3-none-any.whl (16 kB)
#13 5.690 Collecting distlib<1,>=0.3.7
#13 5.694   Downloading distlib-0.3.8-py2.py3-none-any.whl (468 kB)
#13 5.701      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 468.9/468.9 KB 93.0 MB/s eta 0:00:00
#13 5.788 Installing collected packages: distlib, platformdirs, filelock, virtualenv
#13 5.973 Successfully installed distlib-0.3.8 filelock-3.15.4 platformdirs-4.2.2 virtualenv-20.26.3
#13 DONE 6.1s

#14 [7/8] COPY host-x86_64/mingw-check/validate-toolstate.sh /scripts/
#14 DONE 0.0s
---
   Compiling home v0.5.9
error[E0599]: no method named `is_success` found for type `bool` in the current scope
##[error]   --> src/core/build_steps/tool.rs:593:18
    |
585 |               let has_changes = !git(Some(&builder.config.src))
586 | |                 .allow_failure()
587 | |                 .run_always()
587 | |                 .run_always()
588 | |                 .args(["diff-index", "--quiet", &commit])
...   |
592 | |                 .run(builder)
593 | |                 .is_success();
    | |_________________|
    |

error[E0599]: no method named `capture_stdout` found for struct `BootstrapCommand` in the current scope
error[E0599]: no method named `capture_stdout` found for struct `BootstrapCommand` in the current scope
##[error]   --> src/utils/helpers.rs:535:35
    |
535 |     let mut git = git(source_dir).capture_stdout();
    |
   ::: src/utils/exec.rs:59:1
    |
59  | pub struct BootstrapCommand {
---
   Compiling bootstrap v0.0.0 (/checkout/src/bootstrap)
error[E0599]: no method named `is_success` found for type `bool` in the current scope
##[error]   --> src/core/build_steps/tool.rs:593:18
    |
585 |               let has_changes = !git(Some(&builder.config.src))
586 | |                 .allow_failure()
587 | |                 .run_always()
587 | |                 .run_always()
588 | |                 .args(["diff-index", "--quiet", &commit])
...   |
592 | |                 .run(builder)
593 | |                 .is_success();
    | |_________________|
    |

error[E0599]: no method named `capture_stdout` found for struct `BootstrapCommand` in the current scope
error[E0599]: no method named `capture_stdout` found for struct `BootstrapCommand` in the current scope
##[error]   --> src/utils/helpers.rs:535:35
    |
535 |     let mut git = git(source_dir).capture_stdout();
    |
   ::: src/utils/exec.rs:59:1
    |
59  | pub struct BootstrapCommand {
---
   Compiling bootstrap v0.0.0 (/checkout/src/bootstrap)
error[E0599]: no method named `is_success` found for type `bool` in the current scope
##[error]   --> src/core/build_steps/tool.rs:593:18
    |
585 |               let has_changes = !git(Some(&builder.config.src))
586 | |                 .allow_failure()
587 | |                 .run_always()
587 | |                 .run_always()
588 | |                 .args(["diff-index", "--quiet", &commit])
...   |
592 | |                 .run(builder)
593 | |                 .is_success();
    | |_________________|
    |

error[E0599]: no method named `capture_stdout` found for struct `BootstrapCommand` in the current scope
error[E0599]: no method named `capture_stdout` found for struct `BootstrapCommand` in the current scope
##[error]   --> src/utils/helpers.rs:535:35
    |
535 |     let mut git = git(source_dir).capture_stdout();
    |
   ::: src/utils/exec.rs:59:1
    |
59  | pub struct BootstrapCommand {
---
   Compiling bootstrap v0.0.0 (/checkout/src/bootstrap)
error[E0599]: no method named `is_success` found for type `bool` in the current scope
##[error]   --> src/core/build_steps/tool.rs:593:18
    |
585 |               let has_changes = !git(Some(&builder.config.src))
586 | |                 .allow_failure()
587 | |                 .run_always()
587 | |                 .run_always()
588 | |                 .args(["diff-index", "--quiet", &commit])
...   |
592 | |                 .run(builder)
593 | |                 .is_success();
    | |_________________|
    |

error[E0599]: no method named `capture_stdout` found for struct `BootstrapCommand` in the current scope
error[E0599]: no method named `capture_stdout` found for struct `BootstrapCommand` in the current scope
##[error]   --> src/utils/helpers.rs:535:35
    |
535 |     let mut git = git(source_dir).capture_stdout();
    |
   ::: src/utils/exec.rs:59:1
    |
59  | pub struct BootstrapCommand {
---
   Compiling bootstrap v0.0.0 (/checkout/src/bootstrap)
error[E0599]: no method named `is_success` found for type `bool` in the current scope
##[error]   --> src/core/build_steps/tool.rs:593:18
    |
585 |               let has_changes = !git(Some(&builder.config.src))
586 | |                 .allow_failure()
587 | |                 .run_always()
587 | |                 .run_always()
588 | |                 .args(["diff-index", "--quiet", &commit])
...   |
592 | |                 .run(builder)
593 | |                 .is_success();
    | |_________________|
    |

error[E0599]: no method named `capture_stdout` found for struct `BootstrapCommand` in the current scope
error[E0599]: no method named `capture_stdout` found for struct `BootstrapCommand` in the current scope
##[error]   --> src/utils/helpers.rs:535:35
    |
535 |     let mut git = git(source_dir).capture_stdout();
    |
   ::: src/utils/exec.rs:59:1
    |
59  | pub struct BootstrapCommand {

@tgross35 tgross35 closed this Jul 20, 2024
@Kobzol
Copy link
Contributor

Kobzol commented Jul 20, 2024

Let's land #127463 first.

@tgross35
Copy link
Contributor Author

tgross35 commented Jul 20, 2024

Thanks, dropped #127463 for now sorry, picked the wrong one before I saw you commented. But Matthias beat me to it anyway.

@tgross35 tgross35 deleted the rollup-d3emtzd branch July 20, 2024 11:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-run-make Area: port run-make Makefiles to rmake.rs A-testsuite Area: The testsuite used to check the correctness of rustc O-solid Operating System: SOLID O-unix Operating system: Unix-like rollup A PR which is a rollup S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-libs Relevant to the library team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.