Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce rustdoc GUI tests flakyness #126436

Conversation

GuillaumeGomez
Copy link
Member

Fixes #126430.

r? @notriddle

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue. labels Jun 13, 2024
@lqd
Copy link
Member

lqd commented Jun 13, 2024

With this great documentation, the fix looks simple to review. If you don't absolutely need notriddle's r+, you can have my r+ once the x86_64-gnu-tools builder is green.

@GuillaumeGomez
Copy link
Member Author

This is very nice of you. 😉 Then let's go!

@bors r=lqd rollup

@bors
Copy link
Contributor

bors commented Jun 13, 2024

📌 Commit 3de874a has been approved by lqd

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jun 13, 2024
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Jun 13, 2024
…akyness, r=lqd

Reduce rustdoc GUI tests flakyness

Fixes rust-lang#126430.

r? `@notriddle`
bors added a commit to rust-lang-ci/rust that referenced this pull request Jun 13, 2024
…iaskrgr

Rollup of 9 pull requests

Successful merges:

 - rust-lang#121216 (Always emit `native-static-libs` note, even if it is empty)
 - rust-lang#123962 (change method resolution to constrain hidden types instead of rejecting method candidates)
 - rust-lang#126285 (`UniqueRc`: support allocators and `T: ?Sized`.)
 - rust-lang#126315 (Add pub struct with allow(dead_code) into worklist)
 - rust-lang#126320 (Avoid ICES after reporting errors on erroneous patterns)
 - rust-lang#126343 (Remove some msys2 utils)
 - rust-lang#126351 (std::unix::fs::link using direct linkat call for Solaris.)
 - rust-lang#126399 (extend the check for LLVM build)
 - rust-lang#126436 (Reduce rustdoc GUI tests flakyness)

r? `@ghost`
`@rustbot` modify labels: rollup
@rust-log-analyzer

This comment has been minimized.

@lqd
Copy link
Member

lqd commented Jun 13, 2024

:/

@GuillaumeGomez
Copy link
Member Author

That's interesting!

@bors r-

@bors bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Jun 13, 2024
@workingjubilee
Copy link
Member

should go next whenever it clears CI
@bors p=14

@lqd
Copy link
Member

lqd commented Jun 13, 2024

r=me when x86_64-gnu-tools is green.

@rust-log-analyzer

This comment has been minimized.

@matthiaskrgr
Copy link
Member

@bors treeclosed=13

@GuillaumeGomez
Copy link
Member Author

GuillaumeGomez commented Jun 13, 2024

I don't understand why the code used in literally all GUI doctests is failing only in this one test, especially since we didn't update the test nor the rustdoc output in quite a long time.

I added some debug information, let's see if it's helpful...

@GuillaumeGomez
Copy link
Member Author

Until I figure out what's wrong with this test, I opened #126445 to remove it so the rust's CI is unblocked.

@rust-log-analyzer
Copy link
Collaborator

The job x86_64-gnu-llvm-17 failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)
#16 exporting to docker image format
#16 sending tarball 25.5s done
#16 DONE 28.4s
##[endgroup]
Setting extra environment values for docker:  --env ENABLE_GCC_CODEGEN=1 --env GCC_EXEC_PREFIX=/usr/lib/gcc/
[CI_JOB_NAME=x86_64-gnu-llvm-17]
---
sccache: Starting the server...
##[group]Configure the build
configure: processing command line
configure: 
configure: build.configure-args := ['--build=x86_64-unknown-linux-gnu', '--llvm-root=/usr/lib/llvm-17', '--enable-llvm-link-shared', '--set', 'rust.thin-lto-import-instr-limit=10', '--set', 'change-id=99999999', '--enable-verbose-configure', '--enable-sccache', '--disable-manage-submodules', '--enable-locked-deps', '--enable-cargo-native-static', '--set', 'rust.codegen-units-std=1', '--set', 'dist.compression-profile=balanced', '--dist-compression-formats=xz', '--disable-dist-src', '--release-channel=nightly', '--enable-debug-assertions', '--enable-overflow-checks', '--enable-llvm-assertions', '--set', 'rust.verify-llvm-ir', '--set', 'rust.codegen-backends=llvm,cranelift,gcc', '--set', 'llvm.static-libstdcpp', '--enable-new-symbol-mangling']
configure: target.x86_64-unknown-linux-gnu.llvm-config := /usr/lib/llvm-17/bin/llvm-config
configure: llvm.link-shared     := True
configure: rust.thin-lto-import-instr-limit := 10
configure: change-id            := 99999999

workingjubilee added a commit to workingjubilee/rustc that referenced this pull request Jun 13, 2024
Remove failing GUI test to stop blocking CI until it is fixed

Until rust-lang#126436 is solved, let's unblock rust CI.

r? `@lqd`
bors added a commit to rust-lang-ci/rust that referenced this pull request Jun 14, 2024
Remove failing GUI test to stop blocking CI until it is fixed

Until rust-lang#126436 is solved, let's unblock rust CI.

r? `@lqd`
@fmease
Copy link
Member

fmease commented Jun 14, 2024

@bors treeclosed-

@bors
Copy link
Contributor

bors commented Jun 14, 2024

☔ The latest upstream changes (presumably #126445) made this pull request unmergeable. Please resolve the merge conflicts.

@workingjubilee
Copy link
Member

no shit, really?

@rust-log-analyzer
Copy link
Collaborator

The job x86_64-gnu-tools failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 25 filtered out; finished in 0.00s

     Running tests/ui.rs (obj/build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/ui-ff8e5f7317669711)
## Running ui tests in tests/pass for aarch64-unknown-linux-gnu
   Compiler: "MIRI_ENV_VAR_TEST"="0" "MIRI_TEMP"="/tmp/miri-uitest-qCqHOo" "RUST_BACKTRACE"="1" /checkout/obj/build/x86_64-unknown-linux-gnu/stage1/bin/miri "--error-format=json" "--sysroot=/checkout/obj/build/x86_64-unknown-linux-gnu/miri-sysroot" "-Dwarnings" "-Dunused" "-Ainternal_features" "-Zui-testing" "--target" "aarch64-unknown-linux-gnu" "--out-dir" OUT_DIR
test result: ok. 339 passed; 14 ignored;

## Running ui tests in tests/pass-dep for aarch64-unknown-linux-gnu
## Running ui tests in tests/pass-dep for aarch64-unknown-linux-gnu
   Compiler: "MIRI_ENV_VAR_TEST"="0" "MIRI_TEMP"="/tmp/miri-uitest-qCqHOo" "RUST_BACKTRACE"="1" /checkout/obj/build/x86_64-unknown-linux-gnu/stage1/bin/miri "--error-format=json" "--sysroot=/checkout/obj/build/x86_64-unknown-linux-gnu/miri-sysroot" "-Dwarnings" "-Dunused" "-Ainternal_features" "-Zui-testing" "--target" "aarch64-unknown-linux-gnu" "--out-dir" OUT_DIR
test result: ok. 27 passed; 5 ignored;

## Running ui tests in tests/panic for aarch64-unknown-linux-gnu
## Running ui tests in tests/panic for aarch64-unknown-linux-gnu
   Compiler: "MIRI_ENV_VAR_TEST"="0" "MIRI_TEMP"="/tmp/miri-uitest-qCqHOo" "RUST_BACKTRACE"="1" /checkout/obj/build/x86_64-unknown-linux-gnu/stage1/bin/miri "--error-format=json" "--sysroot=/checkout/obj/build/x86_64-unknown-linux-gnu/miri-sysroot" "-Dwarnings" "-Dunused" "-Ainternal_features" "-Zui-testing" "--target" "aarch64-unknown-linux-gnu" "--out-dir" OUT_DIR
test result: ok.

## Running ui tests in tests/fail for aarch64-unknown-linux-gnu
## Running ui tests in tests/fail for aarch64-unknown-linux-gnu
   Compiler: "MIRI_ENV_VAR_TEST"="0" "MIRI_TEMP"="/tmp/miri-uitest-qCqHOo" "RUST_BACKTRACE"="1" /checkout/obj/build/x86_64-unknown-linux-gnu/stage1/bin/miri "--error-format=json" "--sysroot=/checkout/obj/build/x86_64-unknown-linux-gnu/miri-sysroot" "-Dwarnings" "-Dunused" "-Ainternal_features" "-Zui-testing" "--target" "aarch64-unknown-linux-gnu" "--out-dir" OUT_DIR
test result: ok. 9 passed;

## Running ui tests in tests/fail-dep for aarch64-unknown-linux-gnu
## Running ui tests in tests/fail-dep for aarch64-unknown-linux-gnu
   Compiler: "MIRI_ENV_VAR_TEST"="0" "MIRI_TEMP"="/tmp/miri-uitest-qCqHOo" "RUST_BACKTRACE"="1" /checkout/obj/build/x86_64-unknown-linux-gnu/stage1/bin/miri "--error-format=json" "--sysroot=/checkout/obj/build/x86_64-unknown-linux-gnu/miri-sysroot" "-Dwarnings" "-Dunused" "-Ainternal_features" "-Zui-testing" "--target" "aarch64-unknown-linux-gnu" "--out-dir" OUT_DIR
test result: ok.

## Running ui tests in tests/native-lib/pass for aarch64-unknown-linux-gnu
## Running ui tests in tests/native-lib/pass for aarch64-unknown-linux-gnu
   Compiler: "MIRI_ENV_VAR_TEST"="0" "MIRI_TEMP"="/tmp/miri-uitest-qCqHOo" "RUST_BACKTRACE"="1" /checkout/obj/build/x86_64-unknown-linux-gnu/stage1/bin/miri "--error-format=json" "--sysroot=/checkout/obj/build/x86_64-unknown-linux-gnu/miri-sysroot" "-Dwarnings" "-Dunused" "-Ainternal_features" "-Zui-testing" "--target" "aarch64-unknown-linux-gnu" "-Zmiri-native-lib=/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-tools/miri-native-lib/libtestlib.so" "--out-dir" OUT_DIR

test result: ok. 1 ignored;


   Compiler: "MIRI_ENV_VAR_TEST"="0" "MIRI_TEMP"="/tmp/miri-uitest-qCqHOo" "RUST_BACKTRACE"="1" /checkout/obj/build/x86_64-unknown-linux-gnu/stage1/bin/miri "--error-format=json" "--sysroot=/checkout/obj/build/x86_64-unknown-linux-gnu/miri-sysroot" "-Dwarnings" "-Dunused" "-Ainternal_features" "-Zui-testing" "--target" "aarch64-unknown-linux-gnu" "-Zmiri-native-lib=/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-tools/miri-native-lib/libtestlib.so" "--out-dir" OUT_DIR
test result: ok.

 finished in 38.232 seconds
##[endgroup]
---
.......... (120/124)
...F       (124/124)


/checkout/tests/rustdoc-gui/help-page.goml help-page... 
[DEBUG] Done!
[DEBUG] EXECUTING (line 8
    from `tests/rustdoc-gui/help-page.goml` line 24
    from `tests/rustdoc-gui/help-page.goml` line 33) "let parseClickVar = await page.$("#settings-menu");
if (parseClickVar === null) { throw '"#settings-menu" not found'; }
await parseClickVar.click();"
[DEBUG] Done!
[DEBUG] EXECUTING (line 10
    from `tests/rustdoc-gui/help-page.goml` line 24
    from `tests/rustdoc-gui/help-page.goml` line 33) "const timeLimit = page.getDefaultTimeout();
const timeAdd = 50;
let allTime = 0;
let parseWaitFor = null;
while (true) {
    parseWaitFor = await page.$("#settings");
    if (parseWaitFor !== null) {
    }
    }
    await new Promise(r => setTimeout(r, timeAdd));
    if (timeLimit === 0) {
    }
    }
    allTime += timeAdd;
    if (allTime >= timeLimit) {
        throw new Error("The following CSS selector \"#settings\" was not found");
}"
[DEBUG] Done!
[DEBUG] EXECUTING (line 12
[DEBUG] EXECUTING (line 12
    from `tests/rustdoc-gui/help-page.goml` line 24
    from `tests/rustdoc-gui/help-page.goml` line 33) "let parseClickVar = await page.$("#theme-ayu");
if (parseClickVar === null) { throw '"#theme-ayu" not found'; }
await parseClickVar.click();"
[DEBUG] Done!
[DEBUG] EXECUTING (line 14
    from `tests/rustdoc-gui/help-page.goml` line 24
    from `tests/rustdoc-gui/help-page.goml` line 33) "const timeLimit = page.getDefaultTimeout();
const timeAdd = 50;
let allTime = 0;
let parseWaitFor = null;
while (true) {
    parseWaitFor = await page.$("#theme-ayu:checked");
    if (parseWaitFor !== null) {
    }
    }
    await new Promise(r => setTimeout(r, timeAdd));
    if (timeLimit === 0) {
    }
    }
    allTime += timeAdd;
    if (allTime >= timeLimit) {
        throw new Error("The following CSS selector \"#theme-ayu:checked\" was not found");
}"
[DEBUG] Done!
[DEBUG] EXECUTING (line 16
[DEBUG] EXECUTING (line 16
    from `tests/rustdoc-gui/help-page.goml` line 24
    from `tests/rustdoc-gui/help-page.goml` line 33) "let parseClickVar = await page.$("#settings-menu");
if (parseClickVar === null) { throw '"#settings-menu" not found'; }
await parseClickVar.click();"
[DEBUG] Done!
[DEBUG] EXECUTING (line 18
    from `tests/rustdoc-gui/help-page.goml` line 24
    from `tests/rustdoc-gui/help-page.goml` line 33) "const timeLimit = page.getDefaultTimeout();
const timeAdd = 50;
let allTime = 0;
let parseWaitForLocalStorage = null;
while (true) {
    parseWaitForLocalStorage = await page.evaluate(() => {
        const errors = [];
        const parseWaitForLocalStorageDict = {
            "rustdoc-theme": "ayu"
        };
        for (const [parseWaitForLocalStorageKey, parseWaitForLocalStorageValue] of Object.entries(parseWaitForLocalStorageDict)) {
            let parseWaitForLocalStorage = window.localStorage.getItem(parseWaitForLocalStorageKey);
            if (parseWaitForLocalStorage != parseWaitForLocalStorageValue) {
                errors.push("localStorage item \"" + parseWaitForLocalStorageKey + "\" (of value \"" + parseWaitForLocalStorageValue + "\") != \"" + parseWaitForLocalStorage + "\"");
        }
        return errors;
    });
    });
    if (parseWaitForLocalStorage.length === 0) {
    }
    }
    await new Promise(r => setTimeout(r, timeAdd));
    if (timeLimit === 0) {
    }
    }
    allTime += timeAdd;
    if (allTime >= timeLimit) {
        const errs = parseWaitForLocalStorage.join(", ");
        throw new Error("The following local storage entries still don't match: [" + errs + "]");
}"
[DEBUG] Done!
[DEBUG] EXECUTING (line 25
[DEBUG] EXECUTING (line 25
    from `tests/rustdoc-gui/help-page.goml` line 33) "if (!arg.showText) {
    throw "`show-text: true` needs to be used before checking for `color` (otherwise the browser doesn't compute it)";
[DEBUG] Done!
[DEBUG] EXECUTING (line 25
[DEBUG] EXECUTING (line 25
    from `tests/rustdoc-gui/help-page.goml` line 33) "const { checkCssProperty } = require('command-helpers.js');

async function checkElem(elem) {
    const nonMatchingProps = [];
    const jsHandle = await elem.evaluateHandle(e => {
        const parseAssertElemCssDict = ["color","background-color","box-shadow"];
        const assertComputedStyle = window.getComputedStyle(e);
        const simple = [];
        const computed = [];
        const keys = [];

        for (const entry of parseAssertElemCssDict) {
            simple.push(e.style[entry]);
            computed.push(assertComputedStyle[entry]);
            keys.push(entry);
        }
        return [keys, simple, computed];
    });
    const [keys, simple, computed] = await jsHandle.jsonValue();
    const values = ["#c5c5c5","#314559","#5c6773 0px -1px 0px 0px inset"];

    for (const [i, key] of keys.entries()) {
        const localErr = [];
        checkCssProperty(key, values[i], simple[i], computed[i], localErr);
            nonMatchingProps.push(...localErr);
    }
    if (nonMatchingProps.length !== 0) {
        const props = nonMatchingProps.join("; ");
        throw "The following errors happened (for selector `#help kbd`): [" + props + "]";
}
}
let parseAssertElemCss = await page.$$("#help kbd");
if (parseAssertElemCss.length === 0) { throw '"#help kbd" not found'; }
for (const elem of parseAssertElemCss) {
    await checkElem(elem);
[DEBUG] Done!
[DEBUG] EXECUTING (line 23
[DEBUG] EXECUTING (line 23
    from `tests/rustdoc-gui/help-page.goml` line 39) "if (arg && arg.debug_log && arg.debug_log.setDebugEnabled) {
    arg.debug_log.setDebugEnabled(true);
} else {
    throw "`debug` command needs an object with a `debug_log` field of `Debug` type!";
[DEBUG] Done!
[DEBUG] EXECUTING (line 8
[DEBUG] EXECUTING (line 8
    from `tests/rustdoc-gui/help-page.goml` line 24
    from `tests/rustdoc-gui/help-page.goml` line 39) "let parseClickVar = await page.$("#settings-menu");
if (parseClickVar === null) { throw '"#settings-menu" not found'; }
await parseClickVar.click();"
[DEBUG] Done!
[DEBUG] EXECUTING (line 10
    from `tests/rustdoc-gui/help-page.goml` line 24
    from `tests/rustdoc-gui/help-page.goml` line 39) "const timeLimit = page.getDefaultTimeout();
const timeAdd = 50;
let allTime = 0;
let parseWaitFor = null;
while (true) {
    parseWaitFor = await page.$("#settings");
    if (parseWaitFor !== null) {
    }
    }
    await new Promise(r => setTimeout(r, timeAdd));
    if (timeLimit === 0) {
    }
    }
    allTime += timeAdd;
    if (allTime >= timeLimit) {
        throw new Error("The following CSS selector \"#settings\" was not found");
}"
[DEBUG] Done!
[DEBUG] EXECUTING (line 12
[DEBUG] EXECUTING (line 12
    from `tests/rustdoc-gui/help-page.goml` line 24
    from `tests/rustdoc-gui/help-page.goml` line 39) "let parseClickVar = await page.$("#theme-dark");
if (parseClickVar === null) { throw '"#theme-dark" not found'; }
await parseClickVar.click();"
[DEBUG] Done!
[DEBUG] EXECUTING (line 14
    from `tests/rustdoc-gui/help-page.goml` line 24
    from `tests/rustdoc-gui/help-page.goml` line 39) "const timeLimit = page.getDefaultTimeout();
const timeAdd = 50;
let allTime = 0;
let parseWaitFor = null;
while (true) {
    parseWaitFor = await page.$("#theme-dark:checked");
    if (parseWaitFor !== null) {
    }
    }
    await new Promise(r => setTimeout(r, timeAdd));
    if (timeLimit === 0) {
    }
    }
    allTime += timeAdd;
    if (allTime >= timeLimit) {
        throw new Error("The following CSS selector \"#theme-dark:checked\" was not found");
}"
[DEBUG] Done!
[DEBUG] EXECUTING (line 16
[DEBUG] EXECUTING (line 16
    from `tests/rustdoc-gui/help-page.goml` line 24
    from `tests/rustdoc-gui/help-page.goml` line 39) "let parseClickVar = await page.$("#settings-menu");
if (parseClickVar === null) { throw '"#settings-menu" not found'; }
await parseClickVar.click();"
[DEBUG] Done!
[DEBUG] EXECUTING (line 18
    from `tests/rustdoc-gui/help-page.goml` line 24
    from `tests/rustdoc-gui/help-page.goml` line 39) "const timeLimit = page.getDefaultTimeout();
const timeAdd = 50;
let allTime = 0;
let parseWaitForLocalStorage = null;
while (true) {
    parseWaitForLocalStorage = await page.evaluate(() => {
        const errors = [];
        const parseWaitForLocalStorageDict = {
            "rustdoc-theme": "dark"
        };
        for (const [parseWaitForLocalStorageKey, parseWaitForLocalStorageValue] of Object.entries(parseWaitForLocalStorageDict)) {
            let parseWaitForLocalStorage = window.localStorage.getItem(parseWaitForLocalStorageKey);
            if (parseWaitForLocalStorage != parseWaitForLocalStorageValue) {
                errors.push("localStorage item \"" + parseWaitForLocalStorageKey + "\" (of value \"" + parseWaitForLocalStorageValue + "\") != \"" + parseWaitForLocalStorage + "\"");
        }
        return errors;
    });
    });
    if (parseWaitForLocalStorage.length === 0) {
    }
    }
    await new Promise(r => setTimeout(r, timeAdd));
    if (timeLimit === 0) {
    }
    }
    allTime += timeAdd;
    if (allTime >= timeLimit) {
        const errs = parseWaitForLocalStorage.join(", ");
        throw new Error("The following local storage entries still don't match: [" + errs + "]");
}"
[DEBUG] Done!
[DEBUG] EXECUTING (line 25
[DEBUG] EXECUTING (line 25
    from `tests/rustdoc-gui/help-page.goml` line 39) "if (!arg.showText) {
    throw "`show-text: true` needs to be used before checking for `color` (otherwise the browser doesn't compute it)";
[DEBUG] Done!
[DEBUG] EXECUTING (line 25
[DEBUG] EXECUTING (line 25
    from `tests/rustdoc-gui/help-page.goml` line 39) "const { checkCssProperty } = require('command-helpers.js');

async function checkElem(elem) {
    const nonMatchingProps = [];
    const jsHandle = await elem.evaluateHandle(e => {
        const parseAssertElemCssDict = ["color","background-color","box-shadow"];
        const assertComputedStyle = window.getComputedStyle(e);
        const simple = [];
        const computed = [];
        const keys = [];

        for (const entry of parseAssertElemCssDict) {
            simple.push(e.style[entry]);
            computed.push(assertComputedStyle[entry]);
            keys.push(entry);
        }
        return [keys, simple, computed];
    });
    const [keys, simple, computed] = await jsHandle.jsonValue();
    const values = ["#000","#fafbfc","#c6cbd1 0px -1px 0px 0px inset"];

    for (const [i, key] of keys.entries()) {
        const localErr = [];
        checkCssProperty(key, values[i], simple[i], computed[i], localErr);
            nonMatchingProps.push(...localErr);
    }
    if (nonMatchingProps.length !== 0) {
        const props = nonMatchingProps.join("; ");
        throw "The following errors happened (for selector `#help kbd`): [" + props + "]";
}
}
let parseAssertElemCss = await page.$$("#help kbd");
if (parseAssertElemCss.length === 0) { throw '"#help kbd" not found'; }
for (const elem of parseAssertElemCss) {
    await checkElem(elem);
[DEBUG] Done!
[DEBUG] EXECUTING (line 23
[DEBUG] EXECUTING (line 23
    from `tests/rustdoc-gui/help-page.goml` line 45) "if (arg && arg.debug_log && arg.debug_log.setDebugEnabled) {
    arg.debug_log.setDebugEnabled(true);
} else {
    throw "`debug` command needs an object with a `debug_log` field of `Debug` type!";
[DEBUG] Done!
[DEBUG] EXECUTING (line 8
[DEBUG] EXECUTING (line 8
    from `tests/rustdoc-gui/help-page.goml` line 24
    from `tests/rustdoc-gui/help-page.goml` line 45) "let parseClickVar = await page.$("#settings-menu");
if (parseClickVar === null) { throw '"#settings-menu" not found'; }
await parseClickVar.click();"
[DEBUG] Done!
[DEBUG] EXECUTING (line 10
    from `tests/rustdoc-gui/help-page.goml` line 24
    from `tests/rustdoc-gui/help-page.goml` line 45) "const timeLimit = page.getDefaultTimeout();
const timeAdd = 50;
let allTime = 0;
let parseWaitFor = null;
while (true) {
    parseWaitFor = await page.$("#settings");
    if (parseWaitFor !== null) {
    }
    }
    await new Promise(r => setTimeout(r, timeAdd));
    if (timeLimit === 0) {
    }
    }
    allTime += timeAdd;
    if (allTime >= timeLimit) {
        throw new Error("The following CSS selector \"#settings\" was not found");
}"
[DEBUG] Done!
[DEBUG] EXECUTING (line 12
[DEBUG] EXECUTING (line 12
    from `tests/rustdoc-gui/help-page.goml` line 24
    from `tests/rustdoc-gui/help-page.goml` line 45) "let parseClickVar = await page.$("#theme-light");
if (parseClickVar === null) { throw '"#theme-light" not found'; }
await parseClickVar.click();"
[DEBUG] Done!
[DEBUG] EXECUTING (line 14
    from `tests/rustdoc-gui/help-page.goml` line 24
    from `tests/rustdoc-gui/help-page.goml` line 45) "const timeLimit = page.getDefaultTimeout();
const timeAdd = 50;
let allTime = 0;
let parseWaitFor = null;
while (true) {
    parseWaitFor = await page.$("#theme-light:checked");
    if (parseWaitFor !== null) {
    }
    }
    await new Promise(r => setTimeout(r, timeAdd));
    if (timeLimit === 0) {
    }
    }
    allTime += timeAdd;
    if (allTime >= timeLimit) {
        throw new Error("The following CSS selector \"#theme-light:checked\" was not found");
}"
[DEBUG] Done!
[DEBUG] EXECUTING (line 16
[DEBUG] EXECUTING (line 16
    from `tests/rustdoc-gui/help-page.goml` line 24
    from `tests/rustdoc-gui/help-page.goml` line 45) "let parseClickVar = await page.$("#settings-menu");
if (parseClickVar === null) { throw '"#settings-menu" not found'; }
await parseClickVar.click();"
[DEBUG] Done!
[DEBUG] EXECUTING (line 18
    from `tests/rustdoc-gui/help-page.goml` line 24
    from `tests/rustdoc-gui/help-page.goml` line 45) "const timeLimit = page.getDefaultTimeout();
const timeAdd = 50;
let allTime = 0;
let parseWaitForLocalStorage = null;
while (true) {
    parseWaitForLocalStorage = await page.evaluate(() => {
        const errors = [];
        const parseWaitForLocalStorageDict = {
            "rustdoc-theme": "light"
        };
        for (const [parseWaitForLocalStorageKey, parseWaitForLocalStorageValue] of Object.entries(parseWaitForLocalStorageDict)) {
            let parseWaitForLocalStorage = window.localStorage.getItem(parseWaitForLocalStorageKey);
            if (parseWaitForLocalStorage != parseWaitForLocalStorageValue) {
                errors.push("localStorage item \"" + parseWaitForLocalStorageKey + "\" (of value \"" + parseWaitForLocalStorageValue + "\") != \"" + parseWaitForLocalStorage + "\"");
        }
        return errors;
    });
    });
    if (parseWaitForLocalStorage.length === 0) {
    }
    }
    await new Promise(r => setTimeout(r, timeAdd));
    if (timeLimit === 0) {
    }
    }
    allTime += timeAdd;
    if (allTime >= timeLimit) {
        const errs = parseWaitForLocalStorage.join(", ");
        throw new Error("The following local storage entries still don't match: [" + errs + "]");
}"
[DEBUG] Done!
[DEBUG] EXECUTING (line 25
[DEBUG] EXECUTING (line 25
    from `tests/rustdoc-gui/help-page.goml` line 45) "if (!arg.showText) {
    throw "`show-text: true` needs to be used before checking for `color` (otherwise the browser doesn't compute it)";
[DEBUG] Done!
[DEBUG] EXECUTING (line 25
[DEBUG] EXECUTING (line 25
    from `tests/rustdoc-gui/help-page.goml` line 45) "const { checkCssProperty } = require('command-helpers.js');

async function checkElem(elem) {
    const nonMatchingProps = [];
    const jsHandle = await elem.evaluateHandle(e => {
        const parseAssertElemCssDict = ["color","background-color","box-shadow"];
        const assertComputedStyle = window.getComputedStyle(e);
        const simple = [];
        const computed = [];
        const keys = [];

        for (const entry of parseAssertElemCssDict) {
            simple.push(e.style[entry]);
            computed.push(assertComputedStyle[entry]);
            keys.push(entry);
        }
        return [keys, simple, computed];
    });
    const [keys, simple, computed] = await jsHandle.jsonValue();
    const values = ["#000","#fafbfc","#c6cbd1 0px -1px 0px 0px inset"];

    for (const [i, key] of keys.entries()) {
        const localErr = [];
        checkCssProperty(key, values[i], simple[i], computed[i], localErr);
            nonMatchingProps.push(...localErr);
    }
    if (nonMatchingProps.length !== 0) {
        const props = nonMatchingProps.join("; ");
        throw "The following errors happened (for selector `#help kbd`): [" + props + "]";
}
}
let parseAssertElemCss = await page.$$("#help kbd");
if (parseAssertElemCss.length === 0) { throw '"#help kbd" not found'; }
for (const elem of parseAssertElemCss) {
   Error: ()
 await checkElem(elem);
[DEBUG] Done!
[DEBUG] Done!
[DEBUG] EXECUTING (line 53) "const url = "file:///checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc-gui/doc/test_docs/index.html";
try {
    await page.goto(url);
} catch(exc) {
    if (exc instanceof arg.puppeteer.ProtocolError) {
        throw "Cannot navigate to invalid URL `" + url + "`";
        throw exc;
    }
}"
[DEBUG] Done!
[DEBUG] Done!
[DEBUG] EXECUTING (line 53) "await arg.browser.overridePermissions(page.url(), arg.permissions);"
[DEBUG] Done!
[DEBUG] EXECUTING (line 54) "const viewport = page.viewport();
viewport.width = 1000;
viewport.height = 1000;
await page.setViewport(viewport);"
[DEBUG] Done!
[DEBUG] EXECUTING (line 55) "try {
if ((await page.$("#help")) === null) { throw '"#help" not found'; }
} catch(e) { return; } throw "assert didn't fail";"
[DEBUG] Done!
[DEBUG] EXECUTING (line 56) "let parseClickVar = await page.$("#help-button > a");
if (parseClickVar === null) { throw '"#help-button > a" not found'; }
await parseClickVar.click();"
[DEBUG] Done!
[DEBUG] EXECUTING (line 57) "const { checkCssProperty } = require('command-helpers.js');

async function checkElem(elem) {
    const nonMatchingProps = [];
    const jsHandle = await elem.evaluateHandle(e => {
        const parseAssertElemCssDict = ["display"];
        const assertComputedStyle = window.getComputedStyle(e);
        const simple = [];
        const computed = [];
        const keys = [];

        for (const entry of parseAssertElemCssDict) {
            simple.push(e.style[entry]);
            computed.push(assertComputedStyle[entry]);
            keys.push(entry);
        }
        return [keys, simple, computed];
    });
    const [keys, simple, computed] = await jsHandle.jsonValue();
    const values = ["block"];

    for (const [i, key] of keys.entries()) {
        const localErr = [];
        checkCssProperty(key, values[i], simple[i], computed[i], localErr);
            nonMatchingProps.push(...localErr);
    }
    if (nonMatchingProps.length !== 0) {
        const props = nonMatchingProps.join("; ");
        throw "The following errors happened (for selector `#help`): [" + props + "]";
}
}
let parseAssertElemCss = await page.$("#help");
if (parseAssertElemCss === null) { throw '"#help" not found'; }
await checkElem(parseAssertElemCss);"
[DEBUG] Done!
[DEBUG] EXECUTING (line 58) "const { checkCssProperty } = require('command-helpers.js');

async function checkElem(elem) {
    const nonMatchingProps = [];
    const jsHandle = await elem.evaluateHandle(e => {
        const parseAssertElemCssDict = ["font-size"];
        const assertComputedStyle = window.getComputedStyle(e);

@GuillaumeGomez
Copy link
Member Author

Yep, that was completely useless. All test commands are run as expected, but the rustdoc JS seems to not be run for some reason... I'll try to see if maybe something changed on the CI side.

GuillaumeGomez added a commit to GuillaumeGomez/rust that referenced this pull request Jun 27, 2024
…=notriddle

Update browser-ui-test version to `0.18.0`

Should help with rust-lang#126436.

r? `@notriddle`
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Jun 27, 2024
…=notriddle

Update browser-ui-test version to `0.18.0`

Should help with rust-lang#126436.

r? ``@notriddle``
jhpratt added a commit to jhpratt/rust that referenced this pull request Jun 28, 2024
…=notriddle

Update browser-ui-test version to `0.18.0`

Should help with rust-lang#126436.

r? ```@notriddle```
bors added a commit to rust-lang-ci/rust that referenced this pull request Jun 28, 2024
…otriddle

Update browser-ui-test version to `0.18.0`

Should help with rust-lang#126436.

r? `@notriddle`
@GuillaumeGomez GuillaumeGomez deleted the reduce-gui-tests-flakyness branch June 28, 2024 11:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Repeated spurious rustdoc test failure: rustdoc-gui/help-page
9 participants