[🐛 Bug]: Selenium uses a browser downloaded by Selenium Manager instead of the system one

[lugi0]

What happened?

After upgrading to Selenium 4.13, my code started warning me that the chromedriver and chrome versions were not matching.

I made sure that my container had matching versions for the browser and webdriver I wanted to use

yum list --installed | grep "chrom"
chromedriver.x86_64                       116.0.5845.96-1.el9              @epel
chromium.x86_64                           116.0.5845.96-1.el9              @epel
chromium-common.x86_64                    116.0.5845.96-1.el9              @epel
libchromaprint.x86_64                     1.5.1-1.el9                      @epel

After much head scratching, I found this page: https://www.selenium.dev/documentation/selenium_manager/#automated-browser-management
It looks like newer version of Selenium come with selenium manager by default, and there's no way to stop Selenium from using the browser(s) managed and cached by Selenium Manager

How can we reproduce the issue?

N/A

Relevant log output

The chromedriver version (116.0.5845.96) detected in PATH at /usr/bin/chromedriver might not be compatible with the detected chrome version (117.0.5938.92); currently, chromedriver 117.0.5938.92 is recommended for chrome 117.*, so it is advised to delete the driver in PATH and retry

Operating System

Fedora 38 / CentOS Stream 9

Selenium version

4.13 / 4.11.2

What are the browser(s) and version(s) where you see this issue?

Chromium 116 / Chrome 117

What are the browser driver(s) and version(s) where you see this issue?

ChromeDriver 116

Are you using Selenium Grid?

No

[github-actions]

@lugi0, thank you for creating this issue. We will troubleshoot it as soon as we can.

---

Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

[titusfortner]

Selenium does not download a browser if it doesn't need to.

Set the browser version in your options class. If it doesn't match what is present, selenium will download it. You can use 'stable' if you want it to keep up dynamically.

[titusfortner]

Just reread the issue.

That log output should not be what you see with 4.13. It should not complain if the major versions match

[lugi0]

that's exactly the problem @titusfortner , I have chromedriver and chromium set up in my container at matching versions, but then Selenium ends up using my chromedriver (116) and its own managed google chrome (117), thus the mismatch and warning.

Is there any way to disable selenium manager? I don't want it to mess with my container!

[lugi0]

Furthermore, I am using Selenium through Robot Framework and its SeleniumLibrary - I am using the headlesschrome alias as the browser I want to open, expecting it to use Chromium (since that's what is available in the system).
When I was using Selenium<4.11, this worked just fine - i.e. calling headlesschrome with Chromium installed would use the Chromium + Chromedriver combo from my system.
Now that I've upgraded to 4.13 (and even with 4.11.z) I keep getting this warning because Selenium assumes I want to use Google Chrome instead of the available Chromium

[lugi0]

I've also tried to set up the environment variables mentioned here: https://www.selenium.dev/documentation/selenium_manager/#configuration
Specifically:

SE_BROWSER_VERSION=$(chromium-browser --version | awk '{split($0,a," "); print a[2]}')
SE_DRIVER_VERSION=$(chromedriver --version | awk '{split($0,a," "); print a[2]}')
SE_BROWSER_PATH=$(which chromium-browser)

But Selenium still tries to use Chrome 117 (even after deleting it from ~/.cache/selenium/chrome/linux64)

[lugi0]

Small update: The environment variables did indeed work, I just forgot to export them before running the code :D
However I would still like to stop Selenium from installing its own browsers by default rather than overriding these variables. Is there a way to do that?

[titusfortner]

It is finding Chrome v117 on your system somewhere, even if it isn't showing up in that list.

You can turn on logging to see where it finds it and you can remove it if you don't want it to use that.

If you want v116, specify "116" as browserVersion in Options class.
If you want to specify the browser location, specify the path as binary in Options class
If you want Selenium to skip using SM all together, specify the driver location in Service class

[lugi0]

It is finding Chrome v117 on your system somewhere, even if it isn't showing up in that list.

It's downloading it by itself in ~/.cache/selenium/chrome

[titusfortner]

where is your chromium installed?
Is it snap? Is it on PATH?

I think Selenium Manager is supposed to locate both Chromium and Chrome, but if it can't find it, then it will download it.

[titusfortner]

Oh, I thought we supported Chromium already, we do not: #12511

To use the installed version of Chromium instead of Chrome for Testing, you will need to pass in the location of the driver in the service class as specified above.

[lugi0]

To give some more food for though, I managed to solve it in my environment, but when running inside an OpenShift container I'm getting a weird error and I wasn't able to trace it back in the selenium-manager code, but you might want to take a look.

I managed to fix this one as well by not using the headlesschrome alias and instead using chrome with the --headless argument among the browser options, but I'm not really sure what the root cause is.
If I try to run the exact same scenario in my local machine (so, headlesschrome, no --headless option, SE_BROWSER_PATH pointing to my Chromium binary) everything works as it should

[titusfortner]

The DevToolsActivePort bug is something known and being worked on by Google, I'm not sure the status.

[lugi0]

However it does look like selenium-manager is triggering a Warning with os error 13, which from what I understand is permission-related, regardless of DevToolsActivePort

15:20:23.395 	DEBUG 	Executing process: /tmp/.cache/pypoetry/virtualenvs/ods-ci-UC6Fl63A-py3.9/lib/python3.9/site-packages/selenium/webdriver/common/linux/selenium-manager --browser chrome --output json 	
15:20:23.989 	WARN 	Exception managing chrome: Permission denied (os error 13)

[titusfortner]

@bonigarcia what is that warning saying? Looks like it continues just fine after.

[bonigarcia]

It is hard to say. This error concerns some operation in the file system, but unfortunately, Rust does not point to the line in which it happens. If this is problematic, we can try to trace the execution of Selenium Manager in that machine using the binary directly.

[lugi0]

Does selenium-manager try to stat, ls or something else on a potential chrome binary/install path?
The only thing I can think of is that since this is running in an OpenShift container (so a very unprivileged user), it might be poking in some folder where it doesn't have read permissions

[lugi0]

Also, would a strace help here?

[titusfortner]

"DevToolsActivePort not found" bug fix: https://bugs.chromium.org/p/chromedriver/issues/detail?id=4403#c35

[bonigarcia]

I created a new version of Selenium Manager to troubleshoot the warning message about os error 13. I believe it is not critical since the driver is found, but in any case, it would be nice to discover the cause.

@lugi0: To trace it, please download the selenium-manager binary (for Linux) from here: selenium-manager_linux-x64

Please execute the following command in the shell using the uncompressed binary and share the output here. Thanks:

RUST_BACKTRACE=1 ./selenium-manager --debug --browser chrome

[bonigarcia]

@lugi0 Please use this new binary: selenium-manager_linux-x64, as follows:

./selenium-manager --debug --browser chrome --backtrace

[kpouget]

Hello @bonigarcia, I ran the command in the Pod, with the latest binary (and the previous one), but it doesn't seem to give any interesting insight :/

[pod/ods-ci-0-9shvp/main] + ./selenium-manager --debug --browser chrome --backtrace

DEBUG	Found chromedriver 117.0.5938.132 in PATH: /usr/bin/chromedriver
DEBUG	Running command: /usr/lib64/chromium-browser/chromium-browser.sh --version
DEBUG	Output: "Chromium 117.0.5938.132 Fedora Project"
DEBUG	Detected browser: chrome 117.0.5938.132
WARN	Exception managing chrome: Permission denied (os error 13)
INFO	Driver path: /usr/bin/chromedriver
INFO	Browser path: /usr/bin/chromium-browser

(link to the execution trace and to the output of selenium-manager)

[lugi0]

@bonigarcia anything else we can do? the backtrace didn't reveal much.
The only thing that I noticed is that it's pointing to a different path for the DEBUG calls compared to the INFO ones

[bonigarcia]

@kpouget Thanks for the test.
@lugi0 Yes, please. I made another modification to the Selenium Manager code to capture the backtrace in case of that warning. Please use the following binary selenium-manager_linux-x64 with this command:

RUST_BACKTRACE=1 ./selenium-manager --trace --browser chrome

[kpouget]

we have a backtrace this time ... but it's not really informative 🙃

TRACE	Checking chromedriver in PATH
DEBUG	Found chromedriver 117.0.5938.132 in PATH: /usr/bin/chromedriver
TRACE	Path /usr/bin/chromium-browser has been canonicalized to /usr/lib64/chromium-browser/chromium-browser.sh
TRACE	Path /usr/bin/chromium-browser has been escaped to /usr/lib64/chromium-browser/chromium-browser.sh
TRACE	Using shell command to find out chrome version
DEBUG	Running command: /usr/lib64/chromium-browser/chromium-browser.sh --version
DEBUG	Output: "Chromium 117.0.5938.132 Fedora Project"
TRACE	The version of chrome is 117.0.5938.132
DEBUG	Detected browser: chrome 117.0.5938.132
DEBUG	Exception managing chrome: Permission denied (os error 13). Backtrace:
   0: <unknown>
   1: <unknown>
   2: <unknown>
   3: <unknown>
   4: <unknown>
   5: <unknown>
   6: <unknown>

INFO	Driver path: /usr/bin/chromedriver
INFO	Browser path: /usr/bin/chromium-browser

[bonigarcia]

@kpouget Thanks a lot for the test. I checked it and discovered the debug symbols were not included in the release artifact, so the gathered backtrace is empty. I changed the build configuration to include the debug symbols.

Please use the following binary selenium-manager_linux-x64 with the following command (notice now the RUST_BACKTRACE=1 should not be necessary):

 ./selenium-manager --debug --browser chrome

[kpouget]

@bonigarcia , eventually, here is the stack \o/

TRACE	Checking chromedriver in PATH
DEBUG	Found chromedriver 117.0.5938.132 in PATH: /usr/bin/chromedriver
TRACE	Path /usr/bin/chromium-browser has been canonicalized to /usr/lib64/chromium-browser/chromium-browser.sh
TRACE	Path /usr/bin/chromium-browser has been escaped to /usr/lib64/chromium-browser/chromium-browser.sh
TRACE	Using shell command to find out chrome version
DEBUG	Running command: /usr/lib64/chromium-browser/chromium-browser.sh --version
DEBUG	Output: "Chromium 117.0.5938.132 Fedora Project"
TRACE	The version of chrome is 117.0.5938.132
DEBUG	Detected browser: chrome 117.0.5938.132
DEBUG	Exception managing chrome: Permission denied (os error 13). Backtrace:
   0: selenium_manager::SeleniumManager::get_cache_path
             at /home/runner/work/selenium/selenium/rust/src/lib.rs:1058:9
   1: <selenium_manager::chrome::ChromeManager as selenium_manager::SeleniumManager>::request_driver_version
             at /home/runner/work/selenium/selenium/rust/src/chrome.rs:278:60
   2: selenium_manager::SeleniumManager::discover_or_download_browser_and_driver_version
             at /home/runner/work/selenium/selenium/rust/src/lib.rs:376:30
   3: selenium_manager::SeleniumManager::setup
             at /home/runner/work/selenium/selenium/rust/src/lib.rs:531:15
   4: selenium_manager::main::{{closure}}
             at /home/runner/work/selenium/selenium/rust/src/main.rs:205:23
      core::result::Result<T,E>::and_then
             at /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/core/src/result.rs:1319:22
      selenium_manager::main
             at /home/runner/work/selenium/selenium/rust/src/main.rs:202:5
   5: core::ops::function::FnOnce::call_once
             at /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/core/src/ops/function.rs:250:5
      std::sys_common::backtrace::__rust_begin_short_backtrace
             at /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/std/src/sys_common/backtrace.rs:135:18
   6: main

INFO	Driver path: /usr/bin/chromedriver
INFO	Browser path: /usr/bin/chromium-browser

[kpouget]

so the permission error comes from there:

    fn get_cache_path(&self) -> Result<PathBuf, Box<dyn Error>> {
        let path = Path::new(&self.get_config().cache_path);
        create_path_if_not_exists(path)?;
        let canon_path = self.canonicalize_path(path.to_path_buf());
        Ok(Path::new(&canon_path).to_path_buf())
    }

create_path_if_not_exists(path)?, even if the lineno points to the line above

@lugi0 saw that this path is ~/.cache/selenium, and indeed in know that in our environment, HOME isn't writable

[bonigarcia]

Thanks for your effort in debugging this, @kpouget and @lugi0.

That's definitively the source of the problem. The home is supposed to be writable since Selenium Manager stores the downloaded drivers, browsers, and even the metadata (i.e., the discovered versions made by requesting online repositories). So, in your case, it is just a warning since both the driver and browser are in the PATH.

[kpouget]

The home is supposed to be writable since Selenium Manager stores the downloaded drivers, browsers, and even the metadata (i.e., the discovered versions made by requesting online repositories)

in our case, everything is prefetched during the image build, so Selenium Manager has nothing to store in HOME. Proof is that everything works well without the write access. Maybe this warning could be muted with something like

if not can_write(path): 
  logging.info("Cache is not writable, not using it")

This use case might be common in the case of container execution

[bonigarcia]

@kpouget I created a new Selenium Manager binary that handles a non-writable cache path.

Please use the following binary selenium-manager_linux-x64 with the following command:

 ./selenium-manager --debug --browser chrome

[titusfortner]

Closing as did not get further response. Please open a new issue if the problem still exists.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.