feat: Proof of concept: Use ubi as a library instead of as a binary

[autarch]

I took a stab at this using the library-ize branch of ubi. It seems to work from my testing locally, and the test suite also passed locally.

[autarch]

I honestly don't understand the internals of mise well enough to know if this PR is done or not.

[jdx]

it seems to be failing but it's unclear why:

E2E ./backend/test_ubi_token
  $ GITHUB_TOKEN=foobar mise install -f ubi:goreleaser/goreleaser@v1.25.0 2>&1
  Error: [GITHUB_TOKEN=foobar mise install -f ubi:goreleaser/goreleaser@v1.25.0 2>&1] command failed with status 1
  ./backend/test_ubi_token: 0s
  Error: exited with status code 1
  Test environment can be examined in /tmp/test_ubi_token.JQmoym
mise [test:coverage] exited with code 1
Error: Final attempt failed. Child_process exited with error code 1

[autarch]

it seems to be failing but it's unclear why:

E2E ./backend/test_ubi_token
  $ GITHUB_TOKEN=foobar mise install -f ubi:goreleaser/goreleaser@v1.25.0 2>&1
  Error: [GITHUB_TOKEN=foobar mise install -f ubi:goreleaser/goreleaser@v1.25.0 2>&1] command failed with status 1
  ./backend/test_ubi_token: 0s
  Error: exited with status code 1
  Test environment can be examined in /tmp/test_ubi_token.JQmoym
mise [test:coverage] exited with code 1
Error: Final attempt failed. Child_process exited with error code 1

This test replaces the ubi binary with a shell script that echoes back the GitHub token. It's testing that various env vars which contain a token are ultimately passed to ubi. I'm not sure exactly how to test this with ubi as a library.

[autarch]

Also, I think it'd be nice to add support for some other features, like telling ubi the executable name or telling it that the release it picks must match a certain string. The library-ized code supports this, but it's not clear how this would be done with mise. Would it make sense to add some additional ubi-specific flags, something like:

mise "ubi:houseabsolute/precious@v0.7.1" --ubi-matching musl --ubi-exe precious

The other options I can think of are:

• Extend the syntax for packages that mise accepts to allow for more stuff. But that seems really gross and confusing. I don't think allowing per-backend extensions in that string would make for a nice UI.
• Make mise look for env vars like UBI_MATCHING instead of flags. Personally, I think flags are better since they're self-documenting when you use clap. But on the flip side, adding lots of flags for each different backend could become quite unruly.

[jdx]

mise has support for tool config which should be documented more clearly:

[tools]
# send arbitrary options to the plugin, passed as:
# MISE_TOOL_OPTS__VENV=.venv
python = { version = '3.10', virtualenv = '.venv' }

which could be leveraged here. What's missing is a way to configure this on the command line in part because it's tricky to know how to be able to do that with multiple tools

[autarch]

mise has support for tool config which should be documented more clearly:

[tools]
# send arbitrary options to the plugin, passed as:
# MISE_TOOL_OPTS__VENV=.venv
python = { version = '3.10', virtualenv = '.venv' }

which could be leveraged here. What's missing is a way to configure this on the command line in part because it's tricky to know how to be able to do that with multiple tools

I pushed a commit that adds the ability to set "exe" and "matching" in the config file. I think it's ok if this is only available via the config file for now.

[yodatak]

Hello not sure if linked but i have some issue with existing ubi for now like #2429

[jdx]

@autarch let me know when you think this is in a good place for review (or if it's ready now)

[autarch]

@jdx I think this is mostly ready, but it wouldn't hurt to add some additional e2e tests, I think.

[autarch]

Also, there's an e2e test failure I don't understand - https://github.com/jdx/mise/actions/runs/10548662451/job/29222829250?pr=2290

[jdx]

I think we can remove that e2e test file entirely since it involves mocking out ubi which isn't possible anymore baking the CLI into the codebase. It's running this which fails as you would expect:

mise  ubi-as-library ❯ GITHUB_TOKEN=foobar m i -f ubi:goreleaser/goreleaser@v1.25.0
Error: 
   0: HTTP status client error (401 Unauthorized) for url (https://api.github.com/repos/goreleaser/goreleaser/releases/tags/v1.25.0)

Location:
   src/backend/ubi.rs:85

Version:
   2024.8.12-DEBUG macos-arm64 (23eecad 2024-08-27)

Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.

[jdx]

there's a couple more issues I'd like to see tackled before we include this, generally to avoid adding a ton of crates to increase the build complexity of mise further than it already is:

• tls features houseabsolute/ubi#62 - this one is more important since it would greatly reduce crate count while also adding some flexibility, it should just be a few lines in Cargo.toml but I'm happy to help if you run into any problems since I've dealt with this quite a bit
• renovate houseabsolute/ubi#61 - this one is less important, there is one crate which is out of date but the idea here is this would keep them current in the future—mitigating the # of outdated and therefore duplicated crates in our CLIs

[jdx]

This is going to be a blocker unfortunately. I need a way to set the env vars for the installation but there really isn't a good way to do that. I fear I will need to set the env vars inside the process which is a complete departure from what we have been doing.

An alternative idea I had was to expose ubi as a separate, hidden command but baked into mise, like mise ubi, that way I could execute it like before with env vars specific to just that process.

[jdx]

this may be a similar issue to the openssl things since I needed to statically compile xz in one of the libraries I use in mise (actually xz2 but xz is just a wrapper for xz2)

though perhaps not because in this case we don't need to remove any default features, we're only adding static to the list—cargo may just add that feature to all dependencies using the same version but I need to verify that is the case.

In any case though in ubi I would change this to xz2 just to keep things tidy since that's the actual crate you're using.

EDIT: I confirmed it won't be necessary to expose the static feature from xz2: https://doc.rust-lang.org/cargo/reference/features.html#feature-unification, you should directly use xz2 but that isn't going to block me or anything.