Checking in .bin.wast

[titzer]

In maintaining https://github.com/titzer/wizard-engine, we made the decision to only parse .wast tests where the modules themselves are in binary, i.e. .bin.wast. This has been very helpful in avoiding both the work and potential incompatibilities with implementing our own parser for the entire text format. To date, Wizard's strategy has been to check out the spec repo and proposal repos, building their respective interpreters, and then generate the .bin.wast.

As it turns out, as proposals either stagnate or get their tests merged into the main spec, their respective repos get stuck with their respective toolchains (to wit, old versions of Ocaml, ocamlbuild, or dune).

I was wondering if there is any appetite for building the spec interpreter for each of the tracked proposals and then also building the .bin.wast outputs of them, and checking them in here automatically as part of the CI.

[alexcrichton]

Personally I like the idea of lowering the barrier to entry to run the tests here, although if CI goes so far as to convert to *.bin.wast I'd say that it might be better to go as far as something like the output of wast2json where it's binaries + json which is, in principle, even easier to parser. That assumes that using your local language's json parser is likely easier than writing an s-expression parser, even if you're just parsing binary modules and not each individual instruction for example.

To me though there's a lot of downsides in these ideas though. Building the spec interpreter in CI here is probably going to be very nontrivial (especially if each repo needs its own version of ocaml/dune/etc). Running something like wast2json is also nontrivial because it means that some tool (e.g. wast2json itself or some other tool like wasm-tools json-from-wast) has to support proposals before being integrated with this repository, which also isn't great. And there's also the issue of the exact json schema etc etc.

One strong upside of this repo as-is is that the maintenance burden is quite low. Postprocessing *.wast is going to increase that no matter what and to me it'll be a game of minimizing that cost as much as possible.

[dschuff]

Agreed that this would be nontrivial. I'll also add thought that something like this would be useful for importing wasm tests into WPT (Web Platform Tests). We currently have automation that uses the interpreter to convert the wast tests into binary and generate HTML/JS tests in the WPT style and import them. This currently works for things that get merged into the main branch but it would be nice to get them from proposals too, and there we'd have the same problem of needing either the interpreter from that branch, or another tool like Binaryen or wasm-tools.