Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Headless Chrome test fails #611

Closed
TobiasWehrum opened this issue Mar 30, 2019 · 20 comments · Fixed by #706
Closed

Headless Chrome test fails #611

TobiasWehrum opened this issue Mar 30, 2019 · 20 comments · Fixed by #706
Assignees
Labels
bug Something isn't working PR attached there's a PR open for this issue user report windows
Milestone

Comments

@TobiasWehrum
Copy link

🐛 Bug description

I'm currently following this tutorial: https://rustwasm.github.io/docs/book/game-of-life/testing.html. The default tests/web.rs file looks like this:

//! Test suite for the Web and headless browsers.

#![cfg(target_arch = "wasm32")]

extern crate wasm_bindgen_test;
use wasm_bindgen_test::*;

wasm_bindgen_test_configure!(run_in_browser);

#[wasm_bindgen_test]
fn pass() {
    assert_eq!(1 + 1, 2);
}

The tutorial suggests that I run wasm-pack test --chrome --headless to "confirm that the Rust-generated WebAssembly test is working". Instead it fails with the following output:

C:\dev\Projekte\Rust\Sandbox\wasm-game-of-life>wasm-pack test --chrome --headless
[INFO]: Checking for the Wasm target...
    Finished dev [unoptimized + debuginfo] target(s) in 0.42s
[INFO]: Installing wasm-bindgen...
    Finished dev [unoptimized + debuginfo] target(s) in 0.20s
     Running target\wasm32-unknown-unknown\debug\deps\wasm_game_of_life-18a0250534315977.wasm
no tests to run!
     Running target\wasm32-unknown-unknown\debug\deps\web-1bc24e2bd87521b3.wasm
Running headless tests in Chrome with `C:\Users\Tobias\AppData\Local\.wasm-pack\chromedriver-87c19a493f3a2690\chromedriver.exe`
driver status: exit code: 1
driver stdout:
    Starting ChromeDriver 2.46.628402 (536cd7adbad73a3783fdc2cab92ab2ba7ec361e1) on port 62230
    Only local connections are allowed.
    Please protect ports used by ChromeDriver and related test frameworks to prevent access by malicious code.

error: failed to find element reference in response
error: test failed, to rerun pass '--test web'
Error: Running Wasm tests with wasm-bindgen-test failed
Caused by: failed to execute `cargo test`: exited with exit code: 1

Running the test with wasm-pack test --chrome (and opening the page in chrome) is successful, as is running wasm-pack test --firefox --headless.

🤔 Expected Behavior

The test should work and succeed for chrome headless mode.

👟 Steps to reproduce

  1. cargo generate --git https://github.com/rustwasm/wasm-pack-template
  2. Project Name: wasm-game-of-life
  3. cd wasm-game-of-life
  4. wasm-pack test --chrome --headless

🌍 Your environment

wasm-pack version: 0.7.0
rustc version: 1.33.0
System: Windows 7 64bit
Chrome: 73.0.3683.86 (official build) (64-Bit)

@alexcrichton
Copy link
Contributor

Thanks for the report! Can you rerun the build with RUST_LOG=wasm_bindgen_test_runner and gist the output as well? That should hopefully have some more debug logs to poke around and see what's going on as well

@TobiasWehrum
Copy link
Author

@alexcrichton
Copy link
Contributor

Thanks! From the logs it looks like Chrome may be crashing on startup somehow? Unfortunately though it's not clear why chrome is crashing :(

Perhaps you can google around with that error message and your setup and see if something pops up?

@TobiasWehrum
Copy link
Author

TobiasWehrum commented Apr 2, 2019

Unfortunately I'm a bit stumped on what to do. I tried starting chrome with the parameters outlined in the log (--headless --disable-dev-shm-usage --no-sandbox) which makes Chrome start normally. Just running chromedriver.exe from the terminal also runs without crashes. I feel like I don't have enough (or any, for that matter) domain knowledge to reproduce or debug the situation properly.

edit: To clarify, when I google I do find people with similar problems, but I either don't understand the solutions or just can't apply them because I'm not calling chromedriver/chrome the same way they do. In fact I have no idea how I call it, I'm just using wasm-pack test to call it for me.

@alexcrichton
Copy link
Contributor

I think that chromedriver.exe working may be a false positive as you may have to interact with its json message server to actually get it to spawn Chrome and such.

In general the bug is actually moreso for https://github.com/rustwasm/wasm-bindgen than wasm-pack since the test harness lives in that repository. Perhaps the best way to debug this would be to be able to build a copy of that locally and work with that? If you could poke around the code there and see if inserting some debugging would help, that may help reproduce the issue?

I've sometimes had luck in the past as well passing extra arguments to the webdriver binaries. If there's like a verbose logging option for chromedriver you could try turning that on with CHROMEDRIVER_ARGS which are all passed to the chromedriver executable (and chromedriver may enable passing more arguments to chrome as well)

@TobiasWehrum
Copy link
Author

Thanks for the pointers! I don't know when/if I'll have time to do that and investigate further. I found this bug while checking out Rust/Webassembly, but I don't currently use it anywhere, so other things are more pressing to me.

@alexcrichton
Copy link
Contributor

Ok no worries, thanks for the help so far @TobiasWehrum! We'll leave this open for now to see if others end up having it, and next time I get access to Windows I can try to reproduce myself too

@ashleygwilliams
Copy link
Member

cc @steveklabnik if you have some extra cycles if you could try to reproduce that'd be great

@steveklabnik
Copy link
Contributor

So, I'm getting some weirdness, but different weirdness:

     Running target\wasm32-unknown-unknown\debug\deps\wasm_game_of_life-70fcace57c9fcfb4.wasm
error: could not execute process `C:\Users\Steve Klabnik\AppData\Local\.wasm-pack\wasm-bindgen-58ea13aff44655c4\wasm-bindgen-test-runner.exe "C:\Users\Steve Kl
abnik\tmp\wasm-game-of-life\target\wasm32-unknown-unknown\debug\deps\wasm_game_of_life-70fcace57c9fcfb4.wasm"` (never executed)

Caused by:
  The system cannot find the file specified. (os error 2)
Error: Running Wasm tests with wasm-bindgen-test failed
Caused by: failed to execute `cargo test`: exited with exit code: 101

When running manually:

> & 'C:\Users\Steve Klabnik\AppData\Local\.wasm-pack\wasm-bindgen-58ea13aff44655c4\wasm-bindgen-test-runner.exe' 'C
:\Users\Steve Klabnik\tmp\wasm-game-of-life\target\wasm32-unknown-unknown\debug\deps\wasm_game_of_life-70fcace57c9fcfb4.wasm'
no tests to run!

@steveklabnik
Copy link
Contributor

With that being said, noticing the output above, the "no tests to run" is expected, and when running the correct thing:

> & 'C:\Users\Steve Klabnik\AppData\Local\.wasm-pack\wasm-bindgen-58ea13aff44655c4\wasm-bindgen-test-runner.exe' 'C
:\Users\Steve Klabnik\tmp\wasm-game-of-life\target\wasm32-unknown-unknown\debug\deps\web-e0f73df4e58030b8.wasm'
Running headless tests in Chrome with `chromedriver`
Starting new webdriver session...
DevTools listening on ws://127.0.0.1:57691/devtools/browser/c1d61371-b4ed-450f-9848-7a71584dcedd
running 1 test

test web::pass ... ok

test result: ok. 1 passed; 0 failed; 0 ignored

It seems that the wasm-pack test thing not working is a different bug, but headless does work for me.

@LukasKnuth
Copy link

I have the same problem as @TobiasWehrum with this output:

$ wasm-pack test --chrome --headless
[INFO]: Checking for the Wasm target...
   Compiling proc-macro2 v0.4.30
   Compiling wasm-bindgen-test v0.2.48

    Finished dev [unoptimized + debuginfo] target(s) in 1m 15s
[INFO]: Installing wasm-bindgen...
[INFO]: Getting chromedriver...
    Finished dev [unoptimized + debuginfo] target(s) in 0.18s
     Running target\wasm32-unknown-unknown\debug\deps\wasm_game_of_life-e02c4c95b90047dd.wasm
no tests to run!
     Running target\wasm32-unknown-unknown\debug\deps\web-3f614aae3963f555.wasm
Running headless tests in Chrome with `C:\Users\Lukas\AppData\Local\.wasm-pack\chromedriver-87c19a493f3a2690\chromedriver.exe`
Starting new webdriver session...
DevTools listening on ws://127.0.0.1:53240/devtools/browser/be4deb2e-9471-4087-aa1b-cb92d15c864b
driver status: exit code: 1
driver stdout:
    Starting ChromeDriver 2.46.628402 (536cd7adbad73a3783fdc2cab92ab2ba7ec361e1) on port 53237
    Only local connections are allowed.
    Please protect ports used by ChromeDriver and related test frameworks to prevent access by malicious code.

error: failed to find element reference in response
error: test failed, to rerun pass '--test web'
Error: Running Wasm tests with wasm-bindgen-test failed
Caused by: failed to execute `cargo test`: exited with exit code: 1

This is on Windows 10 Pro 1903 with

  • rustup 1.18.3 (435397f48 2019-05-22)
  • rustc 1.36.0 (a53f9df32 2019-07-03)
  • wasm-pack 0.8.1

Let me know if you need any additional info. For the time being, I'll use the FireFox headless mode.

@MartinKavik
Copy link
Contributor

I think it's because of incompatible ChromeDriver and Chrome version.

Explanation
ChromeDriver 2.46 doesn't support current Chrome version (76) but it's the default version which wasm-pack installs (see https://github.com/rustwasm/wasm-pack/blob/master/src/test/webdriver.rs#L56). I think it also can fail with supported Chrome version if you don't have the latest 2.46.xx driver (@TobiasWehrum's case).
We probably didn't notice it sooner because 2.46 supports more Chrome versions and maybe newer versions (73+) are more sensitive to incompatible driver versions.

Suggestions
wasm-pack should check and install compatible ChromeDriver if necessary. Algorithm for selecting compatible driver is here: https://chromedriver.chromium.org/downloads/version-selection.

Workarounds
wasm-pack is searching for currently installed chromedriver (see https://github.com/rustwasm/wasm-pack/blob/master/src/test/webdriver.rs#L34) so we can use that fact.

install:
  - curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.34.0/install.sh | bash
  - source ~/.nvm/nvm.sh
  - nvm install v10.5
  - npm install -g chromedriver
  - which chromedriver # just for easier debugging

alexcrichton added a commit to alexcrichton/wasm-pack that referenced this issue Aug 19, 2019
Looks like a lot of updates have happened since we first added this!
Let's download the most recent stable version.

Closes rustwasm#611
@alexcrichton
Copy link
Contributor

Nice find @MartinKavik! I've posted #703 to update the version we download

@MartinKavik
Copy link
Contributor

@alexcrichton Can I try to make it more automatic (probably add a Chrome version check or try to download latest stable driver instead of a hard-coded one) before we close this issue?
Because It will be broken again with the new Chrome version.

@alexcrichton
Copy link
Contributor

Oh certainly! I'll let you take care of this since it's probably gonna be more invasive than #703 anyway

@andrewdavidmackenzie
Copy link

Anyone know an easy workaround while we want for #703 to land and be included in an update?

@MartinKavik
Copy link
Contributor

@andrewdavidmackenzie Workarounds in #611 (comment) are not usable for you?
Note: #703 has been closed, I'm waiting for review in #706

@andrewdavidmackenzie
Copy link

I ended up manually installing latest chromedriver via brew and that fixed it, not sure about my travis job yet though. (didn't want to switch to FF just for this....but can do if this doesn't work). Thanks

@JakWai01
Copy link

I have the same issue as @TobiasWehrum had. I just wanted to mention that I am running Linux and this is probably not a windows-only issue then (as suggested by the label).

As a little addendum, my error message looks like this:

[jakob@cone wasm-game-of-life]$ wasm-pack test --chrome --headless
[INFO]: Checking for the Wasm target...
warning: function is never used: `set_panic_hook`
 --> src/utils.rs:1:8
  |
1 | pub fn set_panic_hook() {
  |        ^^^^^^^^^^^^^^
  |
  = note: `#[warn(dead_code)]` on by default

warning: `wasm-game-of-life` (lib) generated 1 warning
warning: `wasm-game-of-life` (lib test) generated 1 warning (1 duplicate)
    Finished dev [unoptimized + debuginfo] target(s) in 0.01s
[INFO]: Installing wasm-bindgen...
warning: function is never used: `set_panic_hook`
 --> src/utils.rs:1:8
  |
1 | pub fn set_panic_hook() {
  |        ^^^^^^^^^^^^^^
  |
  = note: `#[warn(dead_code)]` on by default

warning: `wasm-game-of-life` (lib) generated 1 warning
warning: `wasm-game-of-life` (lib test) generated 1 warning (1 duplicate)
    Finished test [unoptimized + debuginfo] target(s) in 0.01s
     Running unittests (target/wasm32-unknown-unknown/debug/deps/wasm_game_of_life-3e7914766b9f62e7.wasm)
no tests to run!
     Running tests/web.rs (target/wasm32-unknown-unknown/debug/deps/web-2d297e9a21cb4560.wasm)
Set timeout to 20 seconds...
Running headless tests in Chrome on `http://127.0.0.1:39065/`
Try find `webdriver.json` for configure browser's capabilities:
Not found
driver status: signal: 9
driver stdout:
    Starting ChromeDriver 101.0.4951.41 (93c720db8323b3ec10d056025ab95c23a31997c9-refs/branch-heads/4951@{#904}) on port 39065
    Only local connections are allowed.
    Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
    ChromeDriver was started successfully.

Error: non-200 response code: 404
{"value":{"error":"invalid session id","message":"invalid session id","stacktrace":"#0 0x561090d9c533 \u003Cunknown>\n#1 0x561090afb07f \u003Cunknown>\n#2 0x561090b240ab \u003Cunknown>\n#3 0x561090b4eebc \u003Cunknown>\n#4 0x561090b4cb61 \u003Cunknown>\n#5 0x561090b4c397 \u003Cunknown>\n#6 0x561090ad1eb4 \u003Cunknown>\n#7 0x561090ad2d10 \u003Cunknown>\n#8 0x561090de0e1d \u003Cunknown>\n#9 0x561090de4751 \u003Cunknown>\n#10 0x561090dcb07e \u003Cunknown>\n#11 0x561090de5388 \u003Cunknown>\n#12 0x561090dbffe0 \u003Cunknown>\n#13 0x561090ad1a26 \u003Cunknown>\n#14 0x7fcfe4e8f310 \u003Cunknown>\n"}}
error: test failed, to rerun pass '--test web'
Error: Running Wasm tests with wasm-bindgen-test failed
Caused by: failed to execute `cargo test`: exited with exit status: 1
  full command: "cargo" "test" "--target" "wasm32-unknown-unknown"

@mwcz
Copy link

mwcz commented Aug 25, 2023

This just occurred to me on Linux (Fedora 37) with a very similar error:

Running headless tests in Chrome on `http://127.0.0.1:45639/`
Try find `webdriver.json` for configure browser's capabilities:
Not found
driver status: signal: 9 (SIGKILL)
driver stdout:
    Starting ChromeDriver 114.0.5735.90 (386bc09e8f4f2e025eddae123f36f6263096ae49-refs/branch-heads/5735@{#1052}) on port 45639
    Only local connections are allowed.
    Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
    ChromeDriver was started successfully.

I'm not sure why, but dnf install chromedriver fixed the problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working PR attached there's a PR open for this issue user report windows
Projects
None yet
10 participants