Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add throttling to postinstall #93

Closed
wants to merge 6 commits into from

Conversation

MrQubo
Copy link

@MrQubo MrQubo commented Sep 11, 2018

Installing any global packages is currently VERY slow.

It works by spawning the process in the background on every execution of the hook. It executes the reshim only if it's the last hook call within the last 10 seconds.

I've tested this one on Arch Linux. It works well and reduces the execution times for individual hook to the minimum.

You can test it by adding echo ASDF RESHIM NODEJS above asdf reshim command. Then execute npm i -g npm, it will call the hook a few hundred times, but the echo message should be displayed only once (with 10-second delay after last hook call).

@MrQubo
Copy link
Author

MrQubo commented Sep 11, 2018

I think the delay should be reduced. 3 or 2 seconds should work fine. When I tested with 1 second it spawned the reshim more than once.

@MrQubo
Copy link
Author

MrQubo commented Sep 11, 2018

Actually npm i -g npm will spawn hooks for dependencies only if the npm itself will be updated. You can try with some other package with a lot of dependencies like npx. Just uninstall it if you have it installed and install it again.

@@ -1,3 +1,42 @@
#!/usr/bin/env bash

asdf reshim nodejs "$ASDF_INSTALL_VERSION"
set -e
Copy link
Author

@MrQubo MrQubo Sep 18, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We might consider removing set -e.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like being strict when it comes to scripts like this.

Previous implementation had some corner case bugs.
New implementation is much simpler and the comments were added.
This value should still prevent reshim from being called more than once.
set -e

declare LOCK_FILE="/tmp/asdf-nodejs-postinstall.lock.2ac8ccc4"
declare LAST_CALLED_FILE="/tmp/asdf-nodejs-postinstall.last_called.dd05fa02"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These paths should be dynamic. We should probably use mktemp for these files.

Copy link
Author

@MrQubo MrQubo Sep 21, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need one common lock file and one common last_called file. We can check if /tmp/asdf-nodejs-postinstall.last_called.* exists and use the existing one if it does, generating it otherwise.
But we would have to run it in the foreground thus making it a little slower.

Actually, I think we can even safely drop the .2ac8ccc4 and .dd05fa02 suffixes, it's just me being a little paranoic.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes you can get rid of those hashes if there is only going to be at most one file at a time.

@Stratus3D
Copy link
Member

Was this code copied from somewhere? Seems like most of it is pretty generic, except for that one line containing the reshim call. If it was copied from somewhere please cite the source so we can verify the license is compatible with ours.

Overall I think this is great. I'm curious if this could cause problems in shell scripts though. Suppose a shell script installs something and then immediately tries to use it. asdf reshim may have not yet finished. What does everyone else think of these changes?

@MrQubo
Copy link
Author

MrQubo commented Sep 21, 2018

Yes, I'm aware that there will be some delay between installation and reshim. But I only thought about using in it shell prompt. I think it would cause a lot of problems with scripts like your example so don't merge it right now.

And I thought of a possibly better solution. Why not just call reshim in asdf/shims/npm if the second command line argument is one of install, uninstall, update, or the alias of one of those. It would sometimes call reshim when it's not needed, but it would always do when it's needed.
Actually, currently, uninstall does not work and does not cause the reshim, it'll fix this too.

The version in the first commit is based on the idea from this gist. But I've rewritten and simplified the code in the second commit and it uses a little different idea now.

@mohsenkhanpour
Copy link

mohsenkhanpour commented Nov 3, 2018

@MrQubo @Stratus3D If it fixes the asdf reshim problem on install and uninstall and it boosts the general installation speed it is worth giving it a try. It may close several issues.
How stable is this PR? I am willing to test it, if feedback is needed.

Also do you know how NVM handles this issue?

@MrQubo
Copy link
Author

MrQubo commented Nov 7, 2018

A bunch of asdf plugins e.g. haskell, ocaml, python, elixir do not call reshim and mentions that reshim has to be called separately after installing new scripts/packages/binaries.
@Stratus3D How about we do it similarly?

@Stratus3D
Copy link
Member

@MrQubo possibly, or we may stick with the slow current implementation until a better solution is found. I would be interested to know how nvm solves this issue. It seems like this is an issue all the version managers would face.

I'm going to close this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants