compiletest: Deficiencies and pitfalls of //@ normalize-* headers
#126372
Labels
A-compiletest
Area: the compiletest test runner
A-testsuite
Area: The testsuite used to check the correctness of rustc
C-cleanup
Category: PRs that clean code up or issues documenting cleanup.
T-compiler
Relevant to the compiler team, which will review and decide on the PR/issue.
Comments
rustbot
added
needs-triage
jieyouxu
added
C-cleanup
bors
added a commit
to rust-lang-ci/rust
that referenced
this issue
Jul 10, 2024
Require a colon in `//@ normalize-*:` test headers The previous parser for `//@ normalize-*` headers (before rust-lang#126370) was so lax that it did not require `:` after the header name. As a result, the test suite contained a mix of with-colon and without-colon normalize headers, both numbering in the hundreds. This PR updates the without-colon headers to add a colon (matching the style used by other headers), and then updates the parser to make the colon mandatory. (Because the normalization parser only runs *after* the header system identifies a normalize header, this will detect and issue an error for relevant headers that lack the colon.) Addresses one of the points of rust-lang#126372.
bors
added a commit
to rust-lang-ci/rust
that referenced
this issue
Jul 11, 2024
Require a colon in `//@ normalize-*:` test headers The previous parser for `//@ normalize-*` headers (before rust-lang#126370) was so lax that it did not require `:` after the header name. As a result, the test suite contained a mix of with-colon and without-colon normalize headers, both numbering in the hundreds. This PR updates the without-colon headers to add a colon (matching the style used by other headers), and then updates the parser to make the colon mandatory. (Because the normalization parser only runs *after* the header system identifies a normalize header, this will detect and issue an error for relevant headers that lack the colon.) Addresses one of the points of rust-lang#126372.
bors
added a commit
to rust-lang-ci/rust
that referenced
this issue
Jul 11, 2024
Require a colon in `//@ normalize-*:` test headers The previous parser for `//@ normalize-*` headers (before rust-lang#126370) was so lax that it did not require `:` after the header name. As a result, the test suite contained a mix of with-colon and without-colon normalize headers, both numbering in the hundreds. This PR updates the without-colon headers to add a colon (matching the style used by other headers), and then updates the parser to make the colon mandatory. (Because the normalization parser only runs *after* the header system identifies a normalize header, this will detect and issue an error for relevant headers that lack the colon.) Addresses one of the points of rust-lang#126372.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
These headers allow individual tests to define custom output-normalization rules, so that output snapshots are less sensitive to things like platform differences, line numbers, and timestamps.
Unfortunately, they have a few limitations and pitfalls:
//@ normalize-*headers #126370:after the header name, so the test suite contains an inconsistent mix of with-colon and without-colon rules.//@ normalize-*:test headers #126777"characters, which is very awkward for some kinds of output (e.g. JSON).parse_cfg_name_directiveto help read the header name, which is sometimes useful (e.g. for defining separate rules for 32-bit and 64-bit), but most of the time it means that tests have to awkwardly writenormalize-stderr-testinstead of the more intuitivenormalize-stderr.ui-mode tests, and are silently ignored if they appear in other kinds of test.@rustbot label +A-compiletest
The text was updated successfully, but these errors were encountered: