Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Port "wapm install" to Wasmer #3317

Merged
merged 10 commits into from
Nov 17, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ Looking for changes that affect our C API? See the [C API Changelog](lib/c-api/C

## Added

- [#3317](https://github.com/wasmerio/wasmer/pull/3317) Add a `wasmer add` command for adding bindings for a WAPM package to your project (only Python and JavaScript are supported for now)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By the way, you don't need to update the changelog at all in PRs. When doing a release, I just do gh search --prs --merged --repo wasmerio/wasmer and then copy the list since the last release. Otherwise I'd have to look if every PR has included this message. So this commit is nice, but pointless.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah okay. We should probably update the pull request template so we don't tell people to update the changelog.


## Changed

- [#3318](https://github.com/wasmerio/wasmer/pull/3318) Bump the Minimum Supported Rust Version (MSRV) to 1.63
Expand Down
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -748,3 +748,6 @@ install-local: package
test-minimal-versions:
rm -f Cargo.lock
cargo +nightly build --tests -Z minimal-versions --all-features

update-graphql-schema:
curl -sSfL https://registry.wapm.io/graphql/schema.graphql > lib/registry/graphql/schema.graphql
4 changes: 2 additions & 2 deletions lib/cli/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ atty = "0.2"
colored = "2.0"
anyhow = "1.0"
spinner = "0.5.0"
clap = { version = "3.2.22", features = ["derive"] }
clap = { version = "3.2.22", features = ["derive", "env"] }
# For the function names autosuggestion
distance = "0.4"
# For the inspect subcommand
Expand All @@ -56,7 +56,7 @@ log = { version = "0.4", optional = true }
tempfile = "3"
tempdir = "0.3.7"
http_req = { version="^0.8", default-features = false, features = ["rust-tls"], optional = true }
reqwest = { version = "^0.11", default-features = false, feature = ["rustls-tls", "json"], optional = true }
reqwest = { version = "^0.11", default-features = false, features = ["rustls-tls", "json"], optional = true }
serde = { version = "1.0.147", features = ["derive"], optional = true }
dirs = { version = "4.0", optional = true }
serde_json = { version = "1.0", optional = true }
Expand Down
34 changes: 23 additions & 11 deletions lib/cli/src/cli.rs
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,10 @@ use crate::commands::CreateExe;
use crate::commands::CreateObj;
#[cfg(feature = "wast")]
use crate::commands::Wast;
use crate::commands::{Cache, Config, Inspect, List, Login, Run, SelfUpdate, Validate};
use crate::commands::{Add, Cache, Config, Inspect, List, Login, Run, SelfUpdate, Validate};
use crate::error::PrettyError;
use clap::{CommandFactory, ErrorKind, Parser};
use std::fmt;
use std::{fmt, str::FromStr};

#[derive(Parser, Debug)]
#[cfg_attr(
Expand Down Expand Up @@ -150,6 +150,9 @@ enum WasmerCLIOptions {
#[cfg(target_os = "linux")]
#[clap(name = "binfmt")]
Binfmt(Binfmt),

/// Add a WAPM package's bindings to your application.
Add(Add),
}

impl WasmerCLIOptions {
Expand All @@ -173,6 +176,7 @@ impl WasmerCLIOptions {
Self::Wast(wast) => wast.execute(),
#[cfg(target_os = "linux")]
Self::Binfmt(binfmt) => binfmt.execute(),
Self::Add(install) => install.execute(),
}
}
}
Expand Down Expand Up @@ -224,7 +228,7 @@ fn wasmer_main_inner() -> Result<(), anyhow::Error> {
WasmerCLIOptions::Run(Run::from_binfmt_args())
} else {
match command.unwrap_or(&"".to_string()).as_ref() {
"cache" | "compile" | "config" | "create-exe" | "help" | "inspect" | "run"
"add" | "cache" | "compile" | "config" | "create-exe" | "help" | "inspect" | "run"
| "self-update" | "validate" | "wast" | "binfmt" | "list" | "login" => {
WasmerCLIOptions::parse()
}
Expand Down Expand Up @@ -278,7 +282,7 @@ impl fmt::Display for SplitVersion {
#[test]
fn test_split_version() {
assert_eq!(
SplitVersion::new("registry.wapm.io/graphql/python/python").unwrap(),
SplitVersion::parse("registry.wapm.io/graphql/python/python").unwrap(),
SplitVersion {
original: "registry.wapm.io/graphql/python/python".to_string(),
registry: Some("https://registry.wapm.io/graphql".to_string()),
Expand All @@ -288,7 +292,7 @@ fn test_split_version() {
}
);
assert_eq!(
SplitVersion::new("registry.wapm.io/python/python").unwrap(),
SplitVersion::parse("registry.wapm.io/python/python").unwrap(),
SplitVersion {
original: "registry.wapm.io/python/python".to_string(),
registry: Some("https://registry.wapm.io/graphql".to_string()),
Expand All @@ -298,7 +302,7 @@ fn test_split_version() {
}
);
assert_eq!(
SplitVersion::new("namespace/name@version:command").unwrap(),
SplitVersion::parse("namespace/name@version:command").unwrap(),
SplitVersion {
original: "namespace/name@version:command".to_string(),
registry: None,
Expand All @@ -308,7 +312,7 @@ fn test_split_version() {
}
);
assert_eq!(
SplitVersion::new("namespace/name@version").unwrap(),
SplitVersion::parse("namespace/name@version").unwrap(),
SplitVersion {
original: "namespace/name@version".to_string(),
registry: None,
Expand All @@ -318,7 +322,7 @@ fn test_split_version() {
}
);
assert_eq!(
SplitVersion::new("namespace/name").unwrap(),
SplitVersion::parse("namespace/name").unwrap(),
SplitVersion {
original: "namespace/name".to_string(),
registry: None,
Expand All @@ -328,7 +332,7 @@ fn test_split_version() {
}
);
assert_eq!(
SplitVersion::new("registry.wapm.io/namespace/name").unwrap(),
SplitVersion::parse("registry.wapm.io/namespace/name").unwrap(),
SplitVersion {
original: "registry.wapm.io/namespace/name".to_string(),
registry: Some("https://registry.wapm.io/graphql".to_string()),
Expand All @@ -338,13 +342,21 @@ fn test_split_version() {
}
);
assert_eq!(
format!("{}", SplitVersion::new("namespace").unwrap_err()),
format!("{}", SplitVersion::parse("namespace").unwrap_err()),
"Invalid package version: \"namespace\"".to_string(),
);
}

impl SplitVersion {
pub fn new(s: &str) -> Result<SplitVersion, anyhow::Error> {
pub fn parse(s: &str) -> Result<SplitVersion, anyhow::Error> {
s.parse()
}
}

impl FromStr for SplitVersion {
type Err = anyhow::Error;

fn from_str(s: &str) -> Result<Self, Self::Err> {
let command = WasmerCLIOptions::command();
let mut prohibited_package_names = command.get_subcommands().map(|s| s.get_name());

Expand Down
5 changes: 4 additions & 1 deletion lib/cli/src/commands.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
//! The commands available in the Wasmer binary.
mod add;
#[cfg(target_os = "linux")]
mod binfmt;
mod cache;
Expand Down Expand Up @@ -28,7 +29,9 @@ pub use create_exe::*;
pub use create_obj::*;
#[cfg(feature = "wast")]
pub use wast::*;
pub use {cache::*, config::*, inspect::*, list::*, login::*, run::*, self_update::*, validate::*};
pub use {
add::*, cache::*, config::*, inspect::*, list::*, login::*, run::*, self_update::*, validate::*,
};

/// The kind of object format to emit.
#[derive(Debug, Copy, Clone, clap::Parser)]
Expand Down
181 changes: 181 additions & 0 deletions lib/cli/src/commands/add.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,181 @@
use std::process::{Command, Stdio};

use anyhow::{Context, Error};
use clap::Parser;
use wasmer_registry::{Bindings, PartialWapmConfig, ProgrammingLanguage};

use crate::cli::SplitVersion;

/// Add a WAPM package's bindings to your application.
#[derive(Debug, Parser)]
pub struct Add {
/// The registry to fetch bindings from.
#[clap(long, env = "WAPM_REGISTRY")]
registry: Option<String>,
/// Add the JavaScript bindings using "npm install".
#[clap(long, groups = &["bindings", "js"])]
npm: bool,
/// Add the JavaScript bindings using "yarn add".
#[clap(long, groups = &["bindings", "js"])]
yarn: bool,
/// Add the package as a dev-dependency.
#[clap(long, requires = "js")]
dev: bool,
/// Add the Python bindings using "pip install".
#[clap(long, groups = &["bindings", "py"])]
pip: bool,
/// The packages to add (e.g. "wasmer/wasmer-pack@0.5.0" or "python/python")
#[clap(parse(try_from_str))]
packages: Vec<SplitVersion>,
}

impl Add {
/// Execute [`Add`].
pub fn execute(&self) -> Result<(), Error> {
anyhow::ensure!(!self.packages.is_empty(), "No packages specified");

let registry = self
.registry()
.context("Unable to determine which registry to use")?;

let bindings = self.lookup_bindings(&registry)?;

let mut cmd = self.target().command(&bindings);

#[cfg(feature = "debug")]
log::debug!("Running {cmd:?}");

let status = cmd
.stdin(Stdio::null())
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.status()
.with_context(|| {
format!(
"Unable to start \"{:?}\". Is it installed?",
cmd.get_program()
)
})?;

anyhow::ensure!(status.success(), "Command failed: {:?}", cmd);

Ok(())
}

fn lookup_bindings(&self, registry: &str) -> Result<Vec<Bindings>, Error> {
#[cfg(feature = "debug")]
log::debug!("Querying WAPM for the bindings packages");

let mut bindings_to_add = Vec::new();
let language = self.target().language();

for pkg in &self.packages {
let bindings = lookup_bindings_for_package(registry, pkg, &language)
.with_context(|| format!("Unable to find bindings for {pkg}"))?;
bindings_to_add.push(bindings);
}

Ok(bindings_to_add)
}

fn registry(&self) -> Result<String, Error> {
match &self.registry {
Some(r) => Ok(r.clone()),
None => {
let cfg = PartialWapmConfig::from_file()
.map_err(Error::msg)
.context("Unable to load WAPM's config file")?;
Ok(cfg.registry.get_current_registry())
}
}
}

fn target(&self) -> Target {
match (self.pip, self.npm, self.yarn) {
(true, false, false) => Target::Pip,
(false, true, false) => Target::Npm { dev: self.dev },
(false, false, true) => Target::Yarn { dev: self.dev },
_ => unreachable!(
"Clap should ensure at least one item in the \"bindings\" group is specified"
),
}
}
}

fn lookup_bindings_for_package(
registry: &str,
pkg: &SplitVersion,
language: &ProgrammingLanguage,
) -> Result<Bindings, Error> {
let all_bindings =
wasmer_registry::list_bindings(registry, &pkg.package, pkg.version.as_deref())?;

match all_bindings.iter().find(|b| b.language == *language) {
Some(b) => {
#[cfg(feature = "debug")]
{
let Bindings { url, generator, .. } = b;
log::debug!("Found {pkg} bindings generated by {generator} at {url}");
}

Ok(b.clone())
}
None => {
if all_bindings.is_empty() {
anyhow::bail!("The package doesn't contain any bindings");
} else {
todo!();
}
}
}
}

#[derive(Debug, Copy, Clone)]
enum Target {
Pip,
Yarn { dev: bool },
Npm { dev: bool },
}

impl Target {
fn language(self) -> ProgrammingLanguage {
match self {
Target::Pip => ProgrammingLanguage::PYTHON,
Target::Yarn { .. } | Target::Npm { .. } => ProgrammingLanguage::JAVASCRIPT,
}
}

/// Construct a command which we can run to add packages.
///
/// This deliberately runs the command using the OS shell instead of
/// invoking the tool directly. That way we can handle when a version
/// manager (e.g. `nvm` or `asdf`) replaces the tool with a script (e.g.
/// `npm.cmd` or `yarn.ps1`).
///
/// See <https://github.com/wasmerio/wapm-cli/issues/291> for more.
fn command(self, packages: &[Bindings]) -> Command {
let command_line = match self {
Target::Pip => "pip install",
Target::Yarn { dev: true } => "yarn add --dev",
Target::Yarn { dev: false } => "yarn add",
Target::Npm { dev: true } => "npm install --dev",
Target::Npm { dev: false } => "npm install",
};
let mut command_line = command_line.to_string();

for pkg in packages {
command_line.push(' ');
command_line.push_str(&pkg.url);
}

if cfg!(windows) {
let mut cmd = Command::new("cmd");
cmd.arg("/C").arg(command_line);
cmd
} else {
let mut cmd = Command::new("sh");
cmd.arg("-c").arg(command_line);
cmd
}
}
}
2 changes: 1 addition & 1 deletion lib/cli/src/commands/run.rs
Original file line number Diff line number Diff line change
Expand Up @@ -845,7 +845,7 @@ pub(crate) fn try_run_package_or_file(
let package = format!("{}", r.path.display());

let mut is_fake_sv = false;
let mut sv = match SplitVersion::new(&package) {
let mut sv = match SplitVersion::parse(&package) {
Ok(o) => o,
Err(_) => {
let mut fake_sv = SplitVersion {
Expand Down
3 changes: 2 additions & 1 deletion lib/registry/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -23,4 +23,5 @@ wapm-toml = "0.2.0"
tar = "0.4.38"
flate2 = "1.0.24"
semver = "1.0.14"
lzma-rs = "0.2.0"
lzma-rs = "0.2.0"
log = "0.4.17"
22 changes: 22 additions & 0 deletions lib/registry/graphql/queries/get_bindings.graphql
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
query GetBindingsQuery ($name: String!, $version: String = "latest") {
packageVersion: getPackageVersion(name:$name, version:$version) {
bindings {
id
language
url

generator {
packageVersion {
id
version
package {
name
}
}
commandName
}

__typename
}
}
}
Loading