Support for Wasm fuzzers

[vanhauser-thc]

We are implementing an ink! fuzzer (https://polkadot.polkassembly.io/referenda/942).
For this we want to use wasmi to make it easier for developers to use the fuzzer.
As this requires code changes we could fork the repository and people would need to use that, but it would be a better user experience if we can merge this feature upstream into wasmi.

This would be hidden behind a feature flag, as normal wasmi usage should not be affected.

Is that something that would be OK for you to merge?

[Robbepop]

Hi @vanhauser-thc ,

could you please specify what exactly would be required to merge upstream into Wasmi?
Naturally I am hesistant to merge specialized functionality into a general purpose tool such as Wasmi, but if the code that you want to merge upstream is general enough to be useful to others we might want to have it.

For that I need (way) more information about what needs to be done in Wasmi.

[vanhauser-thc]

The plan is to make it generic, so that any wasm code can be fuzzed then with AFL++ and libafl (the best open source fuzzers).
This would be similar to the fuzzing framework for WAVM (https://github.com/fgsect/WAFL), but for wasmi.
Of course to fuzz ink! more glue is required allow for state changes on the chain etc, but that would be outside of the upstream PR.

[vanhauser-thc]

I am not expecting a "yes sure I will merge any pile of shit you throw at me", but rather a "if the code looks reasonable and does not impact my maintenance/dev I am inclined to yes" or "I have no interest at all". :-)

[Robbepop]

How will this be different from the already existing fuzzing infrastructure for Wasmi?
(https://github.com/wasmi-labs/wasmi/tree/main/fuzz)

And can you already describe in what ways Wasmi will be extended to support this new fuzzing?
Also will this live in another repository within the wasmi-labs GitHub organization just like the linked WAFL?

For sure I am inclined to merge code that is well written, general purpose and reasonable that improves quality of Wasmi but be sure that every code that I merge into the wasmi-labs organization and especially into the Wasmi repository will 100% impact my maintenance as the maintainer since the moment I merge those pieces is also the moment where maintenance of the code becomes the responsibility of the repository's maintainers (which is me in this case). ;)

[vanhauser-thc]

https://github.com/wasmi-labs/wasmi/tree/main/fuzz are fuzzers that fuzz wasmi as I read it.

we want to fuzz wasm targets. so the lifted/executed wasm code needs to be instrumented for fuzzing, not wasmi itself. but the code to do that needs to be in wasmi.

I made specifically the WAFL comparison because this basically is a dead fork. They should have upstreamed their modifications to WAVM so WAVM users would be able to fuzz their wasm programs with actual interpreter, not an outdated one.

[Robbepop]

What exactly do you mean with "we want to fuzz wasm targets"? Do you want to fuzz compilers that output Wasm? Do you want to fuzz a specific Wasm target such as wasm32 oder wasm32-wasi or whatever is going to be used for ink! ? Or is it something else?

It would be very helpful to know in what ways Wasmi would be required to be extended. So far I have little understanding of what needs to be done in Wasmi to support your use case. Also why cannot it be a standalone tool such as wasm-instrument or any tool in the Bytecode Alliance's wasm-tools repository?

[vanhauser-thc]

wasmi is executing wasm blobs. we want to fuzz what is happening in the wasm blobs to find security issue in there, and extending this, in ink! contracts.

wasmi will provide coverage instrumentation to fuzzers (e.g. cargo-afl) so that the fuzzing of the wasm blobs happens coverage guided. e.g. by writing coverage information when a jump command happens.

[Robbepop]

How about this: you develop this functionality in a Wasmi fork and once development is finished and polished you can open a PR and we can discuss the proposed inclusions into upstream. Until then it seems to be too vague for me to provide a yes or no. If the utility can be isolated, is generally useful and well engineered to be maintainable then I see no issues for inclusion into upstream. Lots of "if"s I know ... :)

Also: is a more general proper name for this issue: "Support for Wasm fuzzers"?

[Robbepop]

@vanhauser-thc what is your opinion or thinking on the above?

[vanhauser-thc]

We will send a pr once we are ready