Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

We should document more clearly env vars used between bootstrap <-> compiletest <-> test suites #125587

Open
jieyouxu opened this issue May 26, 2024 · 2 comments
Labels
A-compiletest Area: The compiletest test runner A-docs Area: Documentation for any part of the project, including the compiler, standard library, and tools A-testsuite Area: The testsuite used to check the correctness of rustc C-cleanup Category: PRs that clean code up or issues documenting cleanup. E-medium Call for participation: Medium difficulty. Experience needed to fix: Intermediate. E-tedious Call for participation: An issue involves lots of work and is better handled as many small tasks. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Comments

@jieyouxu
Copy link
Member

Location

bootstrap <-> compiletest <-> testsuites

Summary

In tests/run-make (for example, but the lack of docs isn't limited to this particular test suite) we assume a bunch of env vars being available, but AFAIK there isn't any docs on which env vars are assumed available between the bootstrap -> compiletest -> test suites pipeline and for what reasons. It'd be nice to have any kind of docs on them.

The current state is that:

  • bootstrap sets up a bunch of env vars, which are made available to compiletest
  • compiletest assumes bootstrap-set env vars are available
  • compiletest sets up a bunch of env vars, often conditionally available to specific test suites
  • tests inside test suites assume compiletest-set env vars are available.
@jieyouxu jieyouxu added C-cleanup Category: PRs that clean code up or issues documenting cleanup. A-testsuite Area: The testsuite used to check the correctness of rustc T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) A-docs Area: Documentation for any part of the project, including the compiler, standard library, and tools labels May 26, 2024
@rustbot rustbot added the needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. label May 26, 2024
@jieyouxu jieyouxu removed the needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. label May 26, 2024
@jieyouxu jieyouxu added the A-compiletest Area: The compiletest test runner label May 29, 2024
@Noratrieb Noratrieb moved this from Needs Triage / Backlog to Ready in compiletest maintenance and improvements May 29, 2024
@jieyouxu
Copy link
Member Author

jieyouxu commented May 31, 2024

Example:

@jieyouxu jieyouxu added E-medium Call for participation: Medium difficulty. Experience needed to fix: Intermediate. E-tedious Call for participation: An issue involves lots of work and is better handled as many small tasks. labels Oct 17, 2024
@jieyouxu
Copy link
Member Author

Better yet, in bootstrap maybe use a pattern like an env var set with possibly fixed/declared keys that is gradually populated, so that when it's time to call compiletest from bootstrap, the env var set will do cmd.env() in a centralized place. Between compiletest and test suites it's more tricky, but a similar thing can be done to make it easier to track what possible collection of env vars tests can receive.

Additionally, we need to enforce better discipline on what env vars are propagated through, because atm it's a wild west of sometimes just letting through whatever env var the user writes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-compiletest Area: The compiletest test runner A-docs Area: Documentation for any part of the project, including the compiler, standard library, and tools A-testsuite Area: The testsuite used to check the correctness of rustc C-cleanup Category: PRs that clean code up or issues documenting cleanup. E-medium Call for participation: Medium difficulty. Experience needed to fix: Intermediate. E-tedious Call for participation: An issue involves lots of work and is better handled as many small tasks. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
Development

No branches or pull requests

2 participants