Add grammar aware fuzzing

[mediremi]

Add grammar aware fuzzing using cargo-fuzz and arbitrary.

Installing the fuzzer

1. Install rustup
2. Install Rust nightly: rustup toolchain install nightly
3. Install cargo-fuzz: cargo +nightly install cargo-fuzz

Running the fuzzer

make fuzz (or cargo +nightly fuzz run grammar_aware)

Checking fuzzer's coverage

1. rustup component add llvm-tools-preview --toolchain nightly
2. cargo +nightly install rustfilt
3. cargo +nightly fuzz coverage grammar_aware
4. $(find $(rustc --print sysroot) -name llvm-cov) show -Xdemangler=rustfilt fuzz/target/x86_64-unknown-linux-gnu/release/grammar_aware -instr-profile=fuzz/coverage/<fuzz target>/coverage.profdata -show-line-counts-or-regions -show-instantiations

[mediremi]

cc @vlopes11 @Voxelot

[mediremi]

I think that deriving arbitrary::Arbitrary in https://github.com/FuelLabs/fuel-asm/blob/master/src/opcode.rs would eliminate the need for this file

[vlopes11]

@mediremi released!

https://github.com/FuelLabs/fuel-asm/releases/tag/v0.8.0

[mediremi]

@vlopes11 when I run cargo check after updating Cargo.toml to use fuel-asm v0.8, I get the following compilation error:

error[E0277]: the trait bound `error::RuntimeError: From<fuel_tx::PanicReason>` is not satisfied
  --> src/interpreter/flow.rs:21:45
   |
21 |             Err(PanicReason::MemoryOverflow.into())
   |                                             ^^^^ the trait `From<fuel_tx::PanicReason>` is not implemented for `error::RuntimeError`
   |
   = help: the following other types implement trait `From<T>`:
             <error::RuntimeError as From<error::Infallible>>
             <error::RuntimeError as From<fuel_asm::PanicReason>>
             <error::RuntimeError as From<std::io::Error>>
   = note: required because of the requirements on the impl of `Into<error::RuntimeError>` for `fuel_tx::PanicReason`

error[E0277]: the trait bound `error::RuntimeError: From<fuel_tx::PanicReason>` is not satisfied
  --> src/interpreter/flow.rs:23:42
   |
23 |             Err(PanicReason::IllegalJump.into())
   |                                          ^^^^ the trait `From<fuel_tx::PanicReason>` is not implemented for `error::RuntimeError`
   |
   = help: the following other types implement trait `From<T>`:
             <error::RuntimeError as From<error::Infallible>>
             <error::RuntimeError as From<fuel_asm::PanicReason>>
             <error::RuntimeError as From<std::io::Error>>
   = note: required because of the requirements on the impl of `Into<error::RuntimeError>` for `fuel_tx::PanicReason`

error[E0277]: the trait bound `error::RuntimeError: From<fuel_tx::PanicReason>` is not satisfied
  --> src/interpreter/flow.rs:90:52
   |
90 |             return Err(PanicReason::MemoryOverflow.into());
   |                                                    ^^^^ the trait `From<fuel_tx::PanicReason>` is not implemented for `error::RuntimeError`
   |
   = help: the following other types implement trait `From<T>`:
             <error::RuntimeError as From<error::Infallible>>
             <error::RuntimeError as From<fuel_asm::PanicReason>>
             <error::RuntimeError as From<std::io::Error>>
   = note: required because of the requirements on the impl of `Into<error::RuntimeError>` for `fuel_tx::PanicReason`

error[E0308]: mismatched types
   --> src/interpreter/flow.rs:129:75
    |
129 |         let receipt = Receipt::panic(self.internal_contract_or_default(), result, pc, is);
    |                                                                           ^^^^^^ expected struct `fuel_tx::InstructionResult`, found struct `fuel_asm::InstructionResult`
    |
    = note: perhaps two different versions of crate `fuel_asm` are being used?

error[E0277]: the trait bound `error::RuntimeError: From<fuel_tx::PanicReason>` is not satisfied
   --> src/interpreter/flow.rs:145:52
    |
145 |             return Err(PanicReason::MemoryOverflow.into());
    |                                                    ^^^^ the trait `From<fuel_tx::PanicReason>` is not implemented for `error::RuntimeError`
    |
    = help: the following other types implement trait `From<T>`:
              <error::RuntimeError as From<error::Infallible>>
              <error::RuntimeError as From<fuel_asm::PanicReason>>
              <error::RuntimeError as From<std::io::Error>>
    = note: required because of the requirements on the impl of `Into<error::RuntimeError>` for `fuel_tx::PanicReason`

error[E0277]: the trait bound `error::RuntimeError: From<fuel_tx::PanicReason>` is not satisfied
   --> src/interpreter/flow.rs:164:57
    |
164 |             return Err(PanicReason::ContractNotInInputs.into());
    |                                                         ^^^^ the trait `From<fuel_tx::PanicReason>` is not implemented for `error::RuntimeError`
    |
    = help: the following other types implement trait `From<T>`:
              <error::RuntimeError as From<error::Infallible>>
              <error::RuntimeError as From<fuel_asm::PanicReason>>
              <error::RuntimeError as From<std::io::Error>>
    = note: required because of the requirements on the impl of `Into<error::RuntimeError>` for `fuel_tx::PanicReason`

error[E0277]: the trait bound `error::RuntimeError: From<fuel_tx::PanicReason>` is not satisfied
   --> src/interpreter/flow.rs:183:52
    |
183 |             return Err(PanicReason::MemoryOverflow.into());
    |                                                    ^^^^ the trait `From<fuel_tx::PanicReason>` is not implemented for `error::RuntimeError`
    |
    = help: the following other types implement trait `From<T>`:
              <error::RuntimeError as From<error::Infallible>>
              <error::RuntimeError as From<fuel_asm::PanicReason>>
              <error::RuntimeError as From<std::io::Error>>
    = note: required because of the requirements on the impl of `Into<error::RuntimeError>` for `fuel_tx::PanicReason`

error[E0277]: `?` couldn't convert the error to `error::RuntimeError`
   --> src/interpreter/internal.rs:129:58
    |
129 |         self.tx.tx_replace_variable_output(idx, variable)?;
    |                                                          ^ the trait `From<fuel_tx::PanicReason>` is not implemented for `error::RuntimeError`
    |
    = note: the question mark operation (`?`) implicitly performs a conversion on the error value using the `From` trait
    = help: the following other types implement trait `From<T>`:
              <error::RuntimeError as From<error::Infallible>>
              <error::RuntimeError as From<fuel_asm::PanicReason>>
              <error::RuntimeError as From<std::io::Error>>
    = note: required because of the requirements on the impl of `FromResidual<Result<std::convert::Infallible, fuel_tx::PanicReason>>` for `Result<(), error::RuntimeError>`

error[E0277]: `?` couldn't convert the error to `error::RuntimeError`
   --> src/interpreter/internal.rs:136:56
    |
136 |         self.tx.tx_replace_message_output(idx, message)?;
    |                                                        ^ the trait `From<fuel_tx::PanicReason>` is not implemented for `error::RuntimeError`
    |
    = note: the question mark operation (`?`) implicitly performs a conversion on the error value using the `From` trait
    = help: the following other types implement trait `From<T>`:
              <error::RuntimeError as From<error::Infallible>>
              <error::RuntimeError as From<fuel_asm::PanicReason>>
              <error::RuntimeError as From<std::io::Error>>
    = note: required because of the requirements on the impl of `FromResidual<Result<std::convert::Infallible, fuel_tx::PanicReason>>` for `Result<(), error::RuntimeError>`

Some errors have detailed explanations: E0277, E0308.
For more information about an error, try `rustc --explain E0277`.
error: could not compile `fuel-vm` due to 9 previous errors

[vlopes11]

Its because fuel-tx isn't released yet. Its on the way, but you can alternatively patch deps to use fuel-asm 0.8 and fuel-crypto 0.6

[vlopes11]

@mediremi its updated in master. You can merge @ re-run

[mediremi]

Merge conflicts with master have been fixed 👍

Only remaining issue now is that sometimes the fuzzer will discover infinite loops and timeout (which we seem to be running into due to using a gas price of 0). What's the best way to update the grammar_aware.rs fuzz target to use a non-zero gas price? Doing it the naive way and setting gas_price = 1 (for example) results in the panic failed to generate a checked tx: InsufficientFeeAmount { expected: 1, provided: 0 }. Other I guess we could decrease the gas limit significantly (e.g. using 10_000 makes the example below run in only a couple seconds)?

Here's an example of an infinite loop found by the fuzzer:

use fuel_vm::prelude::*;

/*
      FuzzData {
          program: [],
          script_data: [
              52,
              90,
              231,
              181,
              74,
              66,
          ],
      }
*/

#[test]
fn infinite_loop() {
    let mut client = MemoryClient::default();

    let gas_price = 0;
    let gas_limit = 1_000_000;
    let maturity = 0;
    let height = 0;
    let params = ConsensusParameters::DEFAULT;

    let mut tx = Transaction::script(
        gas_price,
        gas_limit,
        maturity,
        vec![],
        vec![
            52,
            90,
            231,
            181,
            74,
            66,
        ],
        vec![],
        vec![],
        vec![],
    )
    .check(height, &params)
    .expect("failed to generate a checked tx");

    client.transact(tx);
}

[vlopes11]

Gas will be charged regardless of the price. The price will only interfere in the post-execution to deduct the consumed gas from the base asset.

Even with a gas price of 0, you still get out of gas normally because the gas is deducted from the global and context gas registers. It is a bit odd that we are in an infinite loop without achieving this exhaustion.

If could be that we have too much gas available, and the fuzzy thread is just timing out before the gas runs out. Every script is expected to end with a RET instruction. Otherwise, it will keep executing until either the gas runs out, or we get to the end of the memory. But these operations should be plain NOOP after the script since the VM memory is always zeroed before execution, and NOOP is the cheapest possible instruction. Maybe this timeout is happening before the execution reaches that memory limit.

I'll take a look into the execution of the script you posted!

[vlopes11]

Confirmed. The gas is being deducted correctly and we are basically executing NOOP(s). Since they are so cheap to execute, it will take a long time until it runs to exhaustion. One example is if you reduce that same script to 1_000 limit - it will run out of gas in seconds. For 1_000_000, in my CPU, it will take roughly 16 minutes until either VM memory overflow (not rust panic), or out of gas.

One alternative is to always put a RET(0x01) in the end of the script.

The gas-tuning is still a TODO so this type of attack is feasible. After we have a prohibitive NOOP cost defined, this should halt

[mediremi]

Since adding RET(0x1) results in some panics no longer appearing (e.g. the ones in #192), I've reduced the gas limit to 1_000 instead.

[vlopes11]

LGTM!

Thanks a lot for the contribution

[mediremi]

Looks like I don't have permission to merge this PR so feel free to merge in your own time 👍

[Voxelot]

This looks great! This will be a helpful reference for us to start doing more advanced fuzzing use-cases.