Ability to specify plugins in a file

[craigfurman]

Is your feature request related to a problem? Please describe.

I work with a lot of projects that check in .tool-versions, because I like using asdf 👍. However, it's not always obvious to new users which plugins they need to install. This is a particular problem for plugins that are not in the official plugin list.

Describe the solution you'd like

We could specify plugins in .asdfrc. I've not looked deeply into parsing asdfrc, or its format, so I'm not sure what form the syntax should take yet.

I had previously thought about specifing plugins in .tool-versions, e.g.

plugin ruby https://github.com/asdf-vm/asdf-ruby
ruby 2.7.0

But, plugins are global to asdf, so perhaps this isn't a good idea in case different .tool-versions files conflict.

Update: I've edited this section to make it clear that a .tool-versions, project-specific solution is not what I proposed, but was something I thought about then discarded.

Describe similar asdf features and why they are not sufficient

Describe workarounds you've considered

Additional context

[T4cC0re]

Just wanting to add, that a plug-in that is referenced in the .tool-versions and is in the upstream library, should be automatically installed when using asdf install. This may or may lead to a combined solution that also takes these repository paths into account.

[craigfurman]

@T4cC0re yeah that might be a good catch for 90% of use cases... defaulting to a plugin that's in the registry, if it exists, is less intrusive than what I'm proposing. Let's see what the maintainers say, and I'm happy to send a PR for either approach.

[Stratus3D]

Related - #240

[craigfurman]

@Stratus3D thanks, I obviously didn't search hard enough for pre-existing issues!

Which of #240, #276, or #313 best represents your current thoughts on the matter? #240 was closed without implementation.

I think specifying plugins in one global place, e.g. .asdfrc (here, #301) is one idea that could work around some of the concerns in those other issues.

There are lots of good points raised in those issues around rough edges with some of the proposed implementations (e.g. #313 (comment)), but a single global plugin location is an iterative improvement on today, where plugins must be installed out-of-band.

[Stratus3D]

I've thought about specifing plugins in .tool-versions, e.g.

plugin ruby https://github.com/asdf-vm/asdf-ruby
ruby 2.7.0

But, plugins are global to asdf, so perhaps this isn't a good idea in case different .tool-versions files conflict.

I think it is safe to say specifying in the .tool-versions file is not viable. Plugins are global and that is by design. Specifying them in project-specific .tool-versions files conflates that design choice and opens us up to conflicts.

Which of #240, #276, or #313 best represents your current thoughts on the matter? #240 was closed without implementation.

See my most recent comment on #313 (comment)

I think specifying plugins in one global place, e.g. .asdfrc (here, #301) is one idea that could work around some of the concerns in those other issues.

Wherever we define the list of plugins, we will effectively be defining a new API in the form of an extremely simple DSL. This is better than a complex DSL, but it is still yet another API that we have to maintain. Quoting what I said on #313 (comment),

I think asdf should provide small, focused, stable API so that other developers can build their own tooling around asdf for things such as environment setup.

There are lots of good points raised in those issues around rough edges with some of the proposed implementations (e.g. #313 (comment)), but a single global plugin location is an iterative improvement on today, where plugins must be installed out-of-band.

I personally don't see any problem with putting this in a script like below (which works today):

#!/bin/bash
# Script to setup my plugins
asdf plugin-add ruby
asdf plugin-add nodesjs https://some.custom/nodejs/repo.git

Or using @colinrymer's approach - #240 (comment)

Both of these have the benefit of reusing asdf's existing API.

[craigfurman]

I think it is safe to say specifying in the .tool-versions file is not viable. Plugins are global and that is by design.

Yeah definitely - I say this in the issue description, but in hindsight I should have just removed that part altogether.

The linked approaches are fine, and as you say are not a lot of work to implement. The problem with them, and I admit it's a minor problem, is that every asdf user must implement pretty much exactly the same orchestration over asdf's API to version control their plugins. That leaves opportunity to centralize that admittedly simple code in the tool itself, to add value for users.

From #313 (comment):

This pull request solves the easy part of the problem - running asdf plugin-add - and doesn't address the hard part that users will have to perform manually - installing dependencies and setting up the environment for the tool

This is fair, and I understand that it's out of scope for asdf to resolve dependencies of plugins. What I'm still not understanding is what that fact has to do with the various proposals for version-controlled, global, plugin lists. Users already have to consider dependencies of plugins, if that plugin has any, before installing them - whether another asdf plugin can provide that dependency or not. I'm not proposing to change that.

[jthegedus]

@craigfurman In the scenario you outline where a new user sees your .tool-versions file in a repo and installs asdf for the first time but then cannot find where to get the plugins from, having a file with plugins at your machine level isn't going to live within the same project whereby the user discovers the .tool-versions file, and versioning that within a single project doesn't seem right.

Discoverabiltiy here is an interesting problem, especially if the plugin source is a company internal mirror/fork of an open-source plugin.

As suggested, proposed solutions fail to identify that plugins are global to the machine, not specific to a project. Two different users with different plugins for the same tool should not particularly matter at the .tool-versions layer as long as the final result is the plugins respect and invoke the correct version from .tool-versions.

IMO we're left with this:

1. plugins are machine global so any soln with that proposes a .tool-versions or project-level soln would need re-work in asdf core to make plugins not global and project level to align with user expectations
2. OSS projects with user A and B using the same tool with different plugins shouldn't matter too much as long as the plugins adhere to the correct tool version and invoke it as expected. IE: users with different plugins for one tool shouldn't matter
3. closed-source projects with mandated internal plugins can't be solved by us at a project-level (re first point).

Communicating device to device configuration is not best done on a project level, so I think the best solution thus far to this problem is to:

1. use comments in your .tool-versions file when not using a default plugin
2. assume the plugin used is from the asdf-plugins repo unless a comment in the .tool-versions specifies otherwise

I don't think we document you can have comments in this file, so perhaps that's our bad there.

Perhaps there's room for us to automatically comment above each .tool-verisons entry when doing an asdf local <name> <version> command?

I think people are giving too much weight to plugins as they're merely a wrapper around downloading, installing and identifying executables to shim. Ideally they should all be doing the same thing (I understand some vary with support of installing various runtimes eg: python and anaconda with the python plugin).

[craigfurman]

@jthegedus you're right that any system-global solution doesn't really address discoverability at the project level. I see now that I re-read my own issue description that the solution I propose doesn't address the stated problem!

On the one hand, any assumptions that an individual .tool-versions makes about what plugin is being used could then only be communicated through docs. On the other hand, if 2 plugins with the same name do unacceptably different things, then arguably one of them is broken. I doubt there are many real-world scenarios with competing plugins here.

Even though it has swung the conversation back to project-based solutions, I do like your idea about automatically installing standard plugins, and falling back on a configured non-standard plugin.

[jthegedus]

So to resolve this issue I think we have two steps to take:

• document we support comments in .tool-versions

2. seriously discuss the suggestion to automatically include comments on asdf local <name> <version>

Thoughts on 2. @asdf-vm/core ?

[jlurena]

I think it is safe to say specifying in the .tool-versions file is not viable. Plugins are global and that is by design.

Yeah definitely - I say this in the issue description, but in hindsight I should have just removed that part altogether.

The linked approaches are fine, and as you say are not a lot of work to implement. The problem with them, and I admit it's a minor problem, is that every asdf user must implement pretty much exactly the same orchestration over asdf's API to version control their plugins. That leaves opportunity to centralize that admittedly simple code in the tool itself, to add value for users.

From #313 (comment):

This pull request solves the easy part of the problem - running asdf plugin-add - and doesn't address the hard part that users will have to perform manually - installing dependencies and setting up the environment for the tool

This is fair, and I understand that it's out of scope for asdf to resolve dependencies of plugins. What I'm still not understanding is what that fact has to do with the various proposals for version-controlled, global, plugin lists. Users already have to consider dependencies of plugins, if that plugin has any, before installing them - whether another asdf plugin can provide that dependency or not. I'm not proposing to change that.

Can be as simple as #240 (comment)

[craigfurman]

@jlurena yep, I currently use an almost-identical script. I would still advocate for adsf doing something to automate plugin installation, despite the friction between local (.tool-versions) and global (installed plugins) state.

[wilwade]

I was unable to find a suggestion for this solution, but I was looking at this problem from this perspective:

If I specify in .tool-versions something that someone else wants to run say locally or in docker, does my adding it to the .tool-versions break their setup?

The answer of course is no and I think that is good.

What I did expect however, which might be able to thread between the lines, is just to output a warning message like:

nodejs 14.16.1 is already installed
ruby 2.7.1 is already installed
rust stable is already installed
postgres 9.4.7 not installed (Missing plugin)

Thoughts? Feels like a way to highlight the potential dependencies without making choices for the user.

[T4cC0re]

Yes, this is one thing that I stumbled over when I first used asdf as a consumer.

Displaying there are missing plugins and what they are leaves the user with clues on how to proceed. I would welcome something like this.

[jthegedus]

Identifying a missing plugin on the local system from the items in .tool-versions would be another improvement we can manage without fundamentally changing asdf

[nikolay]

This issue is a dupe of #240.

[dossy]

I don't know if it's worth opening an entire separate feature request for my idea, but it would be nice to be able to specify the fully qualified plugin path in .tool-versions and have asdf be smart enough to add it as a plugin and install the version specified.

For example:

https://github.com/MetricMike/asdf-awscli.git 2.8.7
terraform 1.3.4

I know people keep going on about how plugins are system global, but this is unambiguous as to exactly what you're asking asdf to install, and removes the "how do we map plugin shortnames to URLs" problem in a very clean and obvious way.

Alternatively, if a plugin shortname is required, then maybe a syntax like this would suffice:

awscli https://github.com/MetricMike/asdf-awscli.git 2.8.7
terraform 1.3.4

[aabouzaid]

Since it looks like there is no solution provided so far in asdf itself, I've created a plugin called asdf-plugin-manager to fix that issue and to manage asdf plugins securely and declaratively.

# .plugin-versions
golang     https://github.com/asdf-community/asdf-golang.git     d8dec15
terraform  https://github.com/asdf-community/asdf-hashicorp.git  c048526

# Add all plugins according to .plugin-versions file
asdf-plugin-manager add-all

Or

# Add named plugin according to .plugin-versions file
asdf-plugin-manager add terraform