Function performance benchmarks

[MatthiasPortzel]

Abstract

NVM is the biggest performance bottleneck in loading a new shell for me, taking significantly more time than anything else. I attempted to determine what bits of the nvm source code were bottlenecking it. This experiment does not attempt to uncover solutions to these problems, merely narrow down their locations.

Environment

I'm using macOS 10.15.6 and zsh 5.7.1. I have only one version of node installed through nvm, v13.7.0. The only global packages I have installed are pm2, npm, and their 620 unique dependencies. Experiments were conducted in /tmp/nvm. All that to say, this should fairly closely mimic a fresh install.

Methodology

I commented out or made changes to the nvm.sh shell script. Then ran it, with

time zsh -c '. "./nvm.sh"'

This was repeated 10 times for each change to ensure no flukes. The mode output is reported below. I don't know what time actually measures or how accurate it is.

Results

1. 0.36.0 release (control)
   • 0.38s user 0.36s system 108% cpu 0.686 total
   • 380ms
2. ljharb/npm_config
   • 0.21s user 0.31s system 110% cpu 0.471 total
   • Saving 170ms
3. Removing the nvm_ensure_version_installed check [1].
   • 0.18s user 0.28s system 112% cpu 0.406 total
   • Saving 30ms
4. Instead of nvm_resolve_local_alias, hardcoded to always return "v13.7.0" (here)
   • 0.08s user 0.10s system 108% cpu 0.166 total
   • Saving 100ms
5. Always return "v13.7.0" in use (here specifically)
   • 0.05s user 0.06s system 114% cpu 0.097 total
   • Saving 30ms

Conclusion

By making a few small changes (which do complete break the functionality), the final run-time drops from 380ms to 50ms. I would consider this theoretical runtime perfectly acceptable, unlike the 500ms that NVM is taking right now. Aside from the current area of focus in npm_config, which is obviously a great improvement, the next step would be looking at the other areas mentioned. Why does nvm_resolve_local_alias default take a 100ms? This is not a question that I am able to answer right now, but seems to be the logical direction to take this issue.

[MatthiasPortzel]

This issue is a follow up to this comment on #2317.

[ljharb]

Awesome, thanks! Once #2317 is landed, I'll take a look at nvm_resolve_local_alias and see how it can be sped up, since that seems to be the slowest thing after the npm config stuff.

[MatthiasPortzel]

Thanks for looking into this!

I've spent some more time on this, but due to the fact that I've never worked with a shell script of this size, my methods may be a little crude. Some more interesting data (that I don't pretend to understand):

• The first time you run nvm.sh, it takes longer than subsequent times, since the environment variables persist. (This makes sense if you assume nvm bypasses the slow nvm_auto.) I have this data so I'm sharing it. It is completely useless. (Using v0.36.0 script.)

  • 0.41s user 0.39s system — No env changes
  • 0.37s user 0.33s system — Setting $NVM_BIN to the correct value before running
  • 0.31s user 0.25s system — Adding node to $PATH
  • 0.29s user 0.19s system — Setting both $NVM_BIN and $PATH

• Resolving aliases is above my pay-grade. A lot of the meat and a lot of the time happens in nvm_version, which nvm_resolve_local_alias calls.

  • nvm_resolve_local_alias "stable" takes about 80ms.
  • nvm_resolve_local_alias "v.13.7" takes about 10ms.
  • nvm_resolve_local_alias default takes 110ms. It first resolves default to stable, then stable to v13.7, then v13.7 to v13.7.0.

What I'm probably going to end up doing is adding node to my path and setting $NVM_BIN in my profile before calling nvm.sh. After #2317 is merged, that should be quick enough that I can add nvm.sh back to my .zshrc.

It might be interesting to see nvm officially support an option like this (pre-computing some look ups either in a cache or an environment variable). Asking people to set a default version in an environment variable seems reasonable, and possibly more maintainable than optimizing the hell out of the default version look-up logic.

[MatthiasPortzel]

Nice to see 0.37.0 out!

• 0.36.0: 0.38s user 0.36s system 108% cpu 0.686 total
• 0.37.0: 0.21s user 0.33s system 110% cpu 0.489 total

This is still pretty long. Adding node to my path manually before calling nvm is the only thing I've found that helps. 0.12s user 0.17s system 104% cpu 0.283 total. (Adding node to the path manually breaks nvm, I do not recommend it.)

It's come to my attention that the time reported from the time built-in is divided into time spent in user-space, time spent in the kernel, and total time. Previously, I assumed user time was an accurate representation of the time I had to wait, but I think total time is more accurate. So we've really gone from 690ms to 490ms.

[HaleTom]

Since we are talking performance here, #2334 #1932 is related.

That issue documents that needing to call nvm current on each prompt is quite expensive (about 0.2 user + sys).

The current version could be stored in a variable (eg $NVM_NODE_VERSION), rather than recalculating it each time the user presses enter.

Currently there is no way to get the current node version without checking a bunch of files.

[MatthiasPortzel]

That's a link to this issue, which issue are you referring to?

[HaleTom]

That's a link to this issue, which issue are you referring to?

Oops, I meant #1932. I edited my initial message to reduce future confusion.

@ljharb asked me to fill out a new issue related to this, but I hope that this reference captures it without the need for the additional overhead.

[MatthiasPortzel]

Okay, interesting.

I'm not sure how you're testing or what your setup looks like, but for me, nvm current seems to take 60ms. It is not terribly slow. Especially compared to startup, which is taking 590ms, or nvm use default, which takes 550ms. If your nvm current is disproportionately slower, that might be worth looking at.

(Bash is much faster than zsh by the way. While zsh starts nvm in 522ms, bash is at 287ms.)

(If you're wondering why I've given numbers for nvm start times anywhere between 490ms to 590ms: I'm not sure what causes these variations. If I run 5 tests in a row they're consistent ±5ms, but they may give ±100ms if I run them tomorrow.)

[eduhenke]

If anyone wants a quick workaround to lower ZSH startup time, and postpone this NVM setup time to the command execution(lazy-loading) you could do something like:

alias nvm_lazy="source /usr/share/nvm/init-nvm.sh && nvm"

That will call the NVM initialization script, only when you call the nvm_lazy alias. I couldn't seem to make it work with the same name "nvm", because the init-nvm script defines a function(or alias) with that same name.

[ljharb]

@eduhenke init-nvm.sh is AUR-specific, and doesn’t exist for everyone else.

[ryenus]

More on the performance issue with some nvm commands:

• nvm_ls vs nvm ls

$ time nvm_ls >/dev/null

real	0m0.159s
user	0m0.081s
sys	0m0.098s

$ time nvm ls >/dev/null

real	0m1.981s
user	0m1.722s
sys	0m2.459s

• nvm_ls_remote vs nvm ls-remote

$ time nvm_ls_remote >/dev/null

real	0m0.971s
user	0m0.147s
sys	0m0.156s

$ time nvm ls-remote >/dev/null

real	0m17.585s
user	0m6.667s
sys	0m6.562s

Especially the latter, it's a bit too much for nvm ls-remote to take almost 20 seconds.

[ryenus]

To improve the performance of nvm ls-remote, how about implementing it in awk?

[ljharb]

@ryenus it already uses awk, but I’m sure it could do so more efficiently since I’m not an awk expert. Happy to review a PR.

[ryenus]

@ryenus it already uses awk, but I’m sure it could do so more efficiently since I’m not an awk expert. Happy to review a PR.

@ljharb here comes the PR: #2827