Add testing infrastructure

[kevaundray]

Problem

It is useful to have some sort of testing harness, so that users can quickly test parts of their program without manually modifying Prover.toml each time.

Solution

A solution would go something along the lines of:

• Having a test attribute
• Possibly also including a Prover structure and a Verifier structure to replace the prover.Toml
• The Prover structure is simply a dictionary of values which we use to call a particular function

Alternatives considered

(Describe any alternative solutions you have considered.)

Additional context

This is a tracking issue and will be used to collect discussion points on this.

[jfecher]

Perhaps we can add a nargo test command which with no arguments could automatically run all functions marked #[test]. Optionally we can allow it to be given the name of a function to run (along with its arguments?) to run that function specifically. For example nargo test foo, nargo test main(2, 3) or nargo test array_add1([1, 2, 3]). On success it prints the result of the function and failure occurs when a constrain is known to fail.

Running a function with specific arguments like this though, we might as well have a full REPL. If we added an interpreter for example, it'd be fairly easy to add step-through debugging functionality as well. We would have to decide whether we would want the interpreter to interpret the monomorphised Ast or the SSA IR.

[kevaundray]

nargo test sounds good I think.

The only obstacle is providing arguments to the test functions in a ux friendly way.

We could create a prover.test.yaml file (I'm not a fan of this) or we could find a way to inline the values that we want to use to call functions in the test function.

[kevaundray]

@jfecher I think we can do the following for now:

fn add(x : Field, y : Field) -> Field {
   x + y
}

#[test]
fn test_add() {
   constrain add(1,2) == 3;
}

#[test]
fn test_add_zero() {
   constrain add(0,0) == 0;
}

Gets translated into:

fn main() {
   test_add();
   test_add_zero();
}

ie we collect all functions marked with the test attribute, create a main function and call each of those functions.

[jfecher]

I'm thinking if we collect all of them into one main then we cannot issue which test failed if a constraint fails to verify.

It'd be more clear and I think not too much more difficult if we treated each of these functions as a separate main and looped through them all, proving and verifying each sequentially. That way when one fails we always know which one.

[kevaundray]

I'm thinking if we collect all of them into one main then we cannot issue which test failed if a constraint fails to verify.

It'd be more clear and I think not too much more difficult if we treated each of these functions as a separate main and looped through them all, proving and verifying each sequentially. That way when one fails we always know which one.

I agree -- sounds good to me. This way would also open up the opportunity to run them in parallel, this was discussed with @vezenov last week.

(We won't be able to run them in parallel now, due to Barretenberg having problems with being externally multithreaded, but this does not invalidate the argument)

[kevaundray]

Tracking issue:

An idea by @cheethas is to have values passed to the testing functions which are randomly generated.

#[test]
fn test_add_zero( y : u32) {
   constrain add(0,y) == y;
}

The above code would generate a random value for y each time the test is ran for fuzzing.

This would be added in a later iteration. If we close this issue, we can open another one with this idea.

[jfecher]

Since this is not something rust supports, I think we should open a new issue for the random values and keep this initial PR issue a bit smaller.

[kevaundray]

Since this is not something rust supports, I think we should open a new issue for the random values and keep this initial PR issue a bit smaller.

Since this is not something rust supports, I think we should open a new issue for the random values and keep this initial PR issue a bit smaller.

I agree, once this is closed, I'll open a new issue