Unbounded memory growth installing java on MacOS Sonoma

[nabn]

Describe the bug
Installing java causes the computer to run out of memory.

To Reproduce
run rtx install java

Expected behavior
Expect rtx to not cause unbounded memory growth and the tool installed.

Installing the same version of java with asdf worked okay (I tried asdf install java zulu-11.66.15_1)

rtx doctor output

rtx version:
  2023.8.0 macos-arm64 (unknown 2023-08-16)

build:
  Target: aarch64-apple-darwin
  Features: DEFAULT, NATIVE_TLS
  Built: Wed, 16 Aug 2023 06:44:47 +0000
  Rust Version: rustc 1.71.1 (eb26296b5 2023-08-03)
  Profile: release

shell:
  /bin/zsh
  zsh 5.9 (x86_64-apple-darwin23.0)

rtx data directory:
  /Users/nabeen/.local/share/rtx

rtx environment variables:
  RTX_SHELL=zsh

settings:
  {"always_keep_download": "false", "always_keep_install": "false", "asdf_compat": "false", "disable_default_shorthands": "false", "disable_tools": "[]", "experimental": "false", "jobs": "4", "legacy_version_file": "true", "legacy_version_file_disable_tools": "[]", "log_level": "INFO", "missing_runtime_behavior": "warn", "plugin_autoupdate_last_check_duration": "10080", "raw": "false", "trusted_config_paths": "[]", "verbose": "false"}

config files:
  /Users/nabeen/.rtx.toml

plugins:
  bun      https://github.com/cometkim/asdf-bun.git#fd84edd
  deno     https://github.com/asdf-community/asdf-deno.git#87442fe
  elixir   https://github.com/asdf-vm/asdf-elixir.git#a4c42e1
  erlang   https://github.com/asdf-vm/asdf-erlang.git#a77ab91
  go       (core)
  gradle   https://github.com/rfrancis/asdf-gradle.git#9a2ca80
  java     https://github.com/rtx-plugins/rtx-java.git#a6fb375
  node     (core)
  python   (core)
  ruby     https://github.com/rtx-plugins/rtx-ruby.git#203001f

toolset:
  ruby@3.2.2

No problems found

Additional context
Command appears stuck after running. I couldn't find anything related on trace level of logging

[jdx]

can you post the output of rtx i java --trace?

[nabn]

Here's the output:

[TRACE] (1) rtx::config: [src/config/mod.rs:263] settings does not exist "/Users/nabeen/.config/rtx/config.toml"
[TRACE] (1) rtx::plugins::rtx_plugin_toml: [src/plugins/rtx_plugin_toml.rs:29] parsing: /Users/nabeen/.local/share/rtx/plugins/ruby/rtx.plugin.toml
[TRACE] (9) rtx::config::config_file::rtx_toml: [src/config/config_file/rtx_toml.rs:59] parsing: /Users/nabeen/.rtx.toml
[TRACE] (3) rtx::config::config_file::tool_versions: [src/config/config_file/tool_versions.rs:57] parsing tool-versions: /Users/nabeen/.tool-versions
[TRACE] (10) rtx::config::config_file::tool_versions: [src/config/config_file/tool_versions.rs:57] parsing tool-versions: /Users/nabeen/Work/<redacted>/Lightning/.tool-versions
[TRACE] (12) rtx::config::config_file::rtx_toml: [src/config/config_file/rtx_toml.rs:59] parsing: /Users/nabeen/Work/<redacted>/Lightning/.rtx.toml
[TRACE] (1) rtx::config: [src/config/mod.rs:68] Settings: Settings {
    experimental: false,
    missing_runtime_behavior: Warn,
    always_keep_download: false,
    always_keep_install: false,
    legacy_version_file: true,
    legacy_version_file_disable_tools: {},
    plugin_autoupdate_last_check_duration: 604800s,
    trusted_config_paths: {},
    verbose: true,
    asdf_compat: false,
    jobs: 4,
    shorthands_file: None,
    disable_default_shorthands: false,
    disable_tools: {},
    log_level: Trace,
    raw: false,
}
[TRACE] (5) rtx::cache: [src/cache.rs:79] reading ~/Library/Caches/rtx/elixir/legacy_filenames.msgpack.z
[DEBUG] (1) rtx::config: [src/config/mod.rs:111] Files: ~/Work/<redacted>/Lightning/.rtx.toml, ~/Work/<redacted>/Lightning/.tool-versions, ~/.rtx.toml, ~/.tool-versions
Installed Plugins: bun, deno, elixir, erlang, gradle, ruby
[DEBUG] (1) rtx::cli: [src/cli/mod.rs:190] rtx i java --trace
[TRACE] (7) rtx::cache: [src/cache.rs:79] reading ~/Library/Caches/rtx/java/remote_versions.msgpack.z
[TRACE] (6) rtx::cache: [src/cache.rs:79] reading ~/Library/Caches/rtx/bun/remote_versions.msgpack.z
[TRACE] (4) rtx::cache: [src/cache.rs:79] reading ~/Library/Caches/rtx/deno/remote_versions.msgpack.z
[TRACE] (3) rtx::cache: [src/cache.rs:79] reading ~/Library/Caches/rtx/gradle/remote_versions.msgpack.z
[DEBUG] (8) rtx::plugins::core::python: [src/plugins/core/python.rs:56] Updating python-build in /Users/nabeen/Library/Caches/rtx/python/pyenv
[DEBUG] (14) rtx::cmd: [src/cmd.rs:80] $ git -C /Users/nabeen/Library/Caches/rtx/python/pyenv branch --show-current
[DEBUG] (14) rtx::git: [src/git.rs:66] current branch for /Users/nabeen/Library/Caches/rtx/python/pyenv: master
[DEBUG] (14) rtx::git: [src/git.rs:24] updating /Users/nabeen/Library/Caches/rtx/python/pyenv to master
[DEBUG] (14) rtx::cmd: [src/cmd.rs:80] $ git -C /Users/nabeen/Library/Caches/rtx/python/pyenv fetch --prune --update-head-ok origin master:master
[DEBUG] (14) rtx::cmd: [src/cmd.rs:80] $ git -C /Users/nabeen/Library/Caches/rtx/python/pyenv rev-parse HEAD
[DEBUG] (14) rtx::git: [src/git.rs:71] current sha for /Users/nabeen/Library/Caches/rtx/python/pyenv: 79a501139fd106b39c6095467930ac506822c4c5
[DEBUG] (14) rtx::cmd: [src/cmd.rs:80] $ git -C /Users/nabeen/Library/Caches/rtx/python/pyenv -c advice.detachedHead=false -c advice.objectNameWarning=false checkout --force master
[DEBUG] (14) rtx::cmd: [src/cmd.rs:80] $ git -C /Users/nabeen/Library/Caches/rtx/python/pyenv rev-parse HEAD
[DEBUG] (14) rtx::git: [src/git.rs:71] current sha for /Users/nabeen/Library/Caches/rtx/python/pyenv: 79a501139fd106b39c6095467930ac506822c4c5
[TRACE] (14) rtx::file: [src/file.rs:68] touch /Users/nabeen/Library/Caches/rtx/python/pyenv
[DEBUG] (17) rtx::cmd: [src/cmd.rs:80] $ /Users/nabeen/Library/Caches/rtx/python/pyenv/plugins/python-build/bin/python-build --definitions
[TRACE] (8) rtx::cache: [src/cache.rs:88] writing ~/Library/Caches/rtx/python/remote_versions.msgpack.z
[DEBUG] (1) rtx::toolset::builder: [src/toolset/builder.rs:63] gradle@8, bun@latest, python@latest, deno@latest, java@zulu-11.66.15_1, ruby@3.2.2
[TRACE] (18) rtx::file: [src/file.rs:40] mkdir -p /Users/nabeen/Library/Caches/rtx/lockfiles
[TRACE] (18) rtx::file: [src/file.rs:23] rm -rf /Users/nabeen/.local/share/rtx/installs/java/zulu-11.66.15_1
[TRACE] (18) rtx::file: [src/file.rs:23] rm -rf /Users/nabeen/.local/share/rtx/downloads/java/zulu-11.66.15_1
[TRACE] (18) rtx::file: [src/file.rs:23] rm -rf /Users/nabeen/Library/Caches/rtx/java/zulu-11.66.15_1
[TRACE] (18) rtx::file: [src/file.rs:40] mkdir -p /Users/nabeen/.local/share/rtx/installs/java/zulu-11.66.15_1
[TRACE] (18) rtx::file: [src/file.rs:40] mkdir -p /Users/nabeen/.local/share/rtx/downloads/java/zulu-11.66.15_1
[TRACE] (18) rtx::file: [src/file.rs:40] mkdir -p /Users/nabeen/Library/Caches/rtx/java/zulu-11.66.15_1
[TRACE] (22) reqwest::blocking::wait: [/Users/brew/Library/Caches/Homebrew/cargo_cache/registry/src/index.crates.io-6f17d22bba15001f/reqwest-0.11.18/src/blocking/wait.rs:49] (ThreadId(22)) park without timeout
[TRACE] (23) reqwest::blocking::client: [/Users/brew/Library/Caches/Homebrew/cargo_cache/registry/src/index.crates.io-6f17d22bba15001f/reqwest-0.11.18/src/blocking/client.rs:1033] (ThreadId(23)) start runtime::block_on
[TRACE] (22) reqwest::blocking::wait: [/Users/brew/Library/Caches/Homebrew/cargo_cache/registry/src/index.crates.io-6f17d22bba15001f/reqwest-0.11.18/src/blocking/wait.rs:16] wait at most 30s
[TRACE] (22) reqwest::blocking::wait: [/Users/brew/Library/Caches/Homebrew/cargo_cache/registry/src/index.crates.io-6f17d22bba15001f/reqwest-0.11.18/src/blocking/wait.rs:42] (ThreadId(22)) park timeout 29.999999333s
[DEBUG] (23) reqwest::connect: [/Users/brew/Library/Caches/Homebrew/cargo_cache/registry/src/index.crates.io-6f17d22bba15001f/reqwest-0.11.18/src/connect.rs:429] starting new connection: https://joschi.github.io/
[TRACE] (18) rtx::file: [src/file.rs:23] rm -rf /Users/nabeen/.local/share/rtx/installs/java/zulu-11.66.15_1
[TRACE] (18) rtx::file: [src/file.rs:23] rm -rf /Users/nabeen/.local/share/rtx/downloads/java/zulu-11.66.15_1
Error:
   0: timed out
   1: timed out waiting on channel

Location:
   src/timeout.rs:16

Version:
   2023.8.0 macos-arm64 (unknown 2023-08-15)

Suggestion: Run with RTX_DEBUG=1 for more information.

Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.

[jdx]

can you run curl https://joschi.github.io/java-metadata/metadata/all.json? Frankly this just looks like you're not being able to connect to that address. lmk if you have a proxy or strange network

[nabn]

I can reach that endpoint okay. Also checked on a couple networks.

[nabn]

just noting that I'm on the public beta of MacOS Sonoma.

[jdx]

ok, maybe this weekend I'll see if I can upgrade and find out what happens. This is definitely a strange one though.

One thing I suppose you could try is installing with cargo install rtx-cli --features rustls and see if this is ssl-related.

[nabn]

cargo install rtx-cli --features rustls then rtx install worked 🎉 🎉

[nabn]

turns out I had multiple versions of rtx installed (cargo + homebrew). I thought I was using the cargo version, but brew version was taking precedence.

[jdx]

huh, were you using jdxcode/brew/rtx or rtx? I think they use different ssl methods.

[nabn]

I believe I was using plain rtx.
wouldn't you expect the tool to crash with an error in case of an ssl issue though?

[jdx]

the code just calls reqwest::blocking::get. It isn't streaming the response or anything. I can't explain why this happened but since the trace is showing it's connecting ssl and using rustls fixed it—I suspect the issue is in how openssl was linked on your machine

[nabn]

oh interesting. Happy to provide additional info if it'll help.

I use 1Password as the IdentityAgent, could that have been messing up?

[vraravam]

Can this be reopened please? My mac (running sonoma beta) also crashed a few times in the past few days, and suddenly I realized that it always ran out of application memory when I was installing this version of java: java@corretto-17.0.8.8.1

I too have installed rtx via homebrew just as rtx and not jdxcode/brew/rtx

[jdx]

I use 1Password as the IdentityAgent, could that have been messing up?

don't think so, this isn't using ssh, it's https

@vraravam try installing with the rustls feature

[vraravam]

thanks for the fast reponse @jdxcode - could you please help how to install with that feature via homebrew?

[jdx]

you can't

[vraravam]

so, is there no other option or a future version where I don't need to depend on cargo? I dont code in rust and dont want to go with the cargo route as a dependency for system tooling installations - would like to stick with homebrew.

[nabn]

@vraravam if you install rtx from the tap (brew install jdxcode/tap/rtx), it should work without issues.

[vraravam]

Thanks for the response @nabn .

Unfortunately, as the screenshot shows:

1. rtx took up 31 GB of RAM (and continued to grow till it took 41 GB, then swapped to disk (I guess - since the RAM usage for rtx went down to MBs), and finally crashed the system once more!
2. As can also be seen in the screenshot, the plugins for node and python were reported to not be found (though it installed them successfully)
3. I followed your advice and uninstalled rtx completely. To be extra sure, I also deleted the ~/.local/share/rtx/ folder before reinstalling via the jdxcode/tap/rtx route in my Brewfile

Any other suggestions please?

[nabn]

when you uninstall all rtx versions (before installing from the tap with brew install jdxcode/tap/rtx), it might be worth checking that there are no other versions lying around, just in case.

Also, not sure what your install_rtx_plugins looks like, but might be worth running rtx install java <version> as its own thing so you can pin point where the problem is.

[vraravam]

1. Yes, I have verified that no other rtx is installed. Also, quit iterm and restarted to ensure nothing is brought along "in memory"
2. that's just a bash script to loop on the occurrences of the .tool-versions file and run rtx plugin add <> on those directories for each line. I am currently trying the individual rtx install java, rtx install python, rtx install ruby as per your suggestions.

Update: ruby and python were successfully installed, java is ongoing right now

[vraravam]

rtx install java seems to be the culprit - RAM usage has gone up into the gigabyte range

[nabn]

does which rtx output the correct installation as you expect?

There might be something else going on. I tried the same steps on my machine (uninstalled with brew uninstall rtx, insured which rtx doesn't output anything, then installed with brew install jdxcode/tap/rtx) and rtx install java@corretto-17.0.8.8.1 worked okay for me.

Not sure what the issue is..

[vraravam]

that's the output of `which rtx` `command rtx` only output the help/man of rtx

[vraravam]

i have libressl installed via brew
but, i also noticed that openssl was installed when i ran rtx install

could that be a problem?

update: removed libressl completely, rebooted, installed openssl via brew, and retried the rtx install java - same result.

[nabn]

that's because of rtx activate zsh in your .zshrc or equivalent config files according to the shell you're using; could try commenting that out when you test it.

I'm not very familiar how rtx works internally, I was just trying to share what I did to get it to work.

[vraravam]

Documenting what worked (still not sure why it worked though!)

ok - finally got it working!!!

After installing rtx, I was running rtx plugin add for all the languages listed in the ~/.tool-versions file. After which, I was running the rtx install without any other arguments. This kept failing for java consistently.

At one point, I noticed that the node and python plugins were not added, but still rtx was able to install those lang versions. Based on this, I again uninstalled rtx, openssl, rebooted the whole OS, reinstalled openssl via homebrew, followed by installation of rtx (also via homebrew). This time around, I did NOT run the rtx plugin add command for the languages. Instead, I just ran rtx install directly after installing rtx. And this time around, all the languages worked!

The output of rtx plugin list --urls returns no lines(!) And all the languages are now working and operational.

My machine is and m1 (arm), running sonoma beta 6 (for future reference in case this matters). I have gone back to using brew install rtx and not brew install jdxcode/tap/rtx btw

[dschuessler]

Can reproduce for Ventura 13.4.

rtx version is from homebrew (no tap).

Commands have been:

1. rtx plugin install java
2. rtx install java

[jdx]

It sounds like the java plugin is the problem, try uninstalling it so you use the core plugin:

rtx plugins uninstall java
rtx install java

[vraravam]

Its surprising that there's a concept of a "core" plugin. I have been using rtx for quite a while now, and i had to always add plugin before trying to install that language. Can this be documented somewhere? Or maybe I missed it?

[jdx]

it is documented: https://github.com/jdxcode/rtx#core-plugins

[jdx]

it looks like we may not be the only ones with this issue: https://news.ycombinator.com/item?id=37498979

Seems Sonoma has some networking issues

[jdx]

I assume this is resolved now since I haven't heard anything new lately