perf, feat: assert.CheckCircuit(...)

[gbotrel]

Description

This PR tackles the issue of heterogeneous tests for circuits and long runs in the CI.
Previously some tests call the test engine directly, sometime we commit tests that only check one curve and one backend to go faster, ... ;

Proposition here is to replace assert.ProverSucceeded assert.ProverFailed assert.SolverSucceeded ... with assert.CheckCircuit.

Through build tags and -short test option, an increasing number of checks are performed;

//	go test -short 				--> testEngineChecks
//	go test 						--> testEngineChecks  + constraintSolverChecks
//	go test -tags=prover_checks 	--> ... + proverChecks
//	go test -tags=release_checks 	--> ... + releaseChecks (solidity, serialization, ...)

This allow a fast dev workflow (test engine only with -short option) and various CI workflows to do extra validations.

This PR also factorize / clean up quite a bit of repeated code (for example for testing serialization round trips).

Last but not least, test.Engine has a BatchInvert method called in logderivarg.go Build() method; it saves 30% of test time since a lot of inverses using big.Int are performed in the test engine. A potential follow up to this PR would be to have the test engine parametrized with field element instead of big.Int.

Fixes # (issue)

This PR also fixes some minor issues;

• groth16.Proof serialization was missing commitment info.
• witness.ReadFrom returned incorrect number of bytes read when deserializing

Breaking changes

• test.NoSerialization() -> test.NoSerializationChecks()

Type of change

• New feature (non-breaking change which adds functionality)
• This change requires a documentation update

How has this been benchmarked?

go test -short ./... up to 90% faster.

Checklist:

• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have added tests that prove my fix is effective or that my feature works
• I did not modify files generated from templates
• golangci-lint does not output errors locally
• New and existing unit tests pass locally with my changes
• Any dependent changes have been merged and published in downstream modules

[ivokub]

Reviewing now.

[ivokub]

Suggested edit:

diff --git a/test/assert.go b/test/assert.go
index 35ed4a45..81c346db 100644
--- a/test/assert.go
+++ b/test/assert.go
@@ -72,7 +72,7 @@ func (assert *Assert) Log(v ...interface{}) {
 	assert.t.Log(v...)
 }
 
-// ProverSucceeded is deprecated: use CheckCircuit instead
+// ProverSucceeded is deprecated: use [Assert.CheckCircuit] instead
 func (assert *Assert) ProverSucceeded(circuit frontend.Circuit, validAssignment frontend.Circuit, opts ...TestingOption) {
 	// copy the options
 	newOpts := make([]TestingOption, len(opts), len(opts)+2)
@@ -82,7 +82,7 @@ func (assert *Assert) ProverSucceeded(circuit frontend.Circuit, validAssignment
 	assert.CheckCircuit(circuit, newOpts...)
 }
 
-// ProverSucceeded is Deprecated use CheckCircuit instead
+// ProverSucceeded is deprecated use [Assert.CheckCircuit] instead
 func (assert *Assert) ProverFailed(circuit frontend.Circuit, invalidAssignment frontend.Circuit, opts ...TestingOption) {
 	// copy the options
 	newOpts := make([]TestingOption, len(opts), len(opts)+2)
@@ -92,7 +92,7 @@ func (assert *Assert) ProverFailed(circuit frontend.Circuit, invalidAssignment f
 	assert.CheckCircuit(circuit, newOpts...)
 }
 
-// SolvingSucceeded is deprecated: use CheckCircuit instead
+// SolvingSucceeded is deprecated: use [Assert.CheckCircuit] instead
 func (assert *Assert) SolvingSucceeded(circuit frontend.Circuit, validWitness frontend.Circuit, opts ...TestingOption) {
 
 	// copy the options
diff --git a/test/assert_checkcircuit.go b/test/assert_checkcircuit.go
index e5bb6b96..f3502ee1 100644
--- a/test/assert_checkcircuit.go
+++ b/test/assert_checkcircuit.go
@@ -14,19 +14,19 @@ import (
 
 // CheckCircuit performs a series of check on the provided circuit.
 //
-//	go test -short 					--> testEngineChecks
-//	go test 						--> testEngineChecks  + constraintSolverChecks
-//	go test -tags=prover_checks 	--> ... + proverChecks
-//	go test -tags=release_checks 	--> ... + releaseChecks (solidity, serialization, ...)
+//	go test -short                  --> testEngineChecks
+//	go test                         --> testEngineChecks  + constraintSolverChecks
+//	go test -tags=prover_checks     --> ... + proverChecks
+//	go test -tags=release_checks    --> ... + releaseChecks (solidity, serialization, ...)
 //
 // Depending on the above flags, the following checks are performed:
-// - the circuit compiles
-// - the circuit can be solved with the test engine
-// - the circuit can be solved with the constraint system solver
-// - the circuit can be solved with the prover
-// - the circuit can be verified with the verifier
-// - the circuit can be verified with gnark-solidity-checker
-// - the circuit, witness, proving and verifying keys can be serialized and deserialized
+//   - the circuit compiles
+//   - the circuit can be solved with the test engine
+//   - the circuit can be solved with the constraint system solver
+//   - the circuit can be solved with the prover
+//   - the circuit can be verified with the verifier
+//   - the circuit can be verified with gnark-solidity-checker
+//   - the circuit, witness, proving and verifying keys can be serialized and deserialized
 func (assert *Assert) CheckCircuit(circuit frontend.Circuit, opts ...TestingOption) {
 	// get the testing configuration
 	opt := assert.options(opts...)

[ivokub]

I think the approach is really good. A few questions about additional behaviour control options (which I'm not sure are worth it).

Also added a few suggestions to make godoc output look nicer.

[github-actions]

Summary

✅ Passed: 5693
❌ Failed: 0
🚧 Skipped: 21

🚧 Skipped

• TestContributionSerialization (github.com/consensys/gnark/backend/groth16/bls12-377/mpcsetup)
• TestSetupCircuit (github.com/consensys/gnark/backend/groth16/bls12-377/mpcsetup)
• TestContributionSerialization (github.com/consensys/gnark/backend/groth16/bls12-381/mpcsetup)
• TestSetupCircuit (github.com/consensys/gnark/backend/groth16/bls12-381/mpcsetup)
• TestContributionSerialization (github.com/consensys/gnark/backend/groth16/bls24-317/mpcsetup)
• TestSetupCircuit (github.com/consensys/gnark/backend/groth16/bls24-317/mpcsetup)
• TestContributionSerialization (github.com/consensys/gnark/backend/groth16/bls24-315/mpcsetup)
• TestSetupCircuit (github.com/consensys/gnark/backend/groth16/bls24-315/mpcsetup)
• TestContributionSerialization (github.com/consensys/gnark/backend/groth16/bn254/mpcsetup)
• TestContributionSerialization (github.com/consensys/gnark/backend/groth16/bw6-633/mpcsetup)
• TestSetupCircuit (github.com/consensys/gnark/backend/groth16/bw6-633/mpcsetup)
• TestContributionSerialization (github.com/consensys/gnark/backend/groth16/bw6-761/mpcsetup)
• TestSetupCircuit (github.com/consensys/gnark/backend/groth16/bw6-761/mpcsetup)
• TestCircuitInclusionProof (github.com/consensys/gnark/examples/rollup)
• TestCircuitUpdateAccount (github.com/consensys/gnark/examples/rollup)
• TestCircuitFull (github.com/consensys/gnark/examples/rollup)
• TestFrobeniusFp12 (github.com/consensys/gnark/std/algebra/native/fields_bls12377)
• TestMulByNonResidueFp2 (github.com/consensys/gnark/std/algebra/native/fields_bls12377)
• TestMulByFp2Fp6 (github.com/consensys/gnark/std/algebra/native/fields_bls12377)
• TestIssue348UnconstrainedLimbs (github.com/consensys/gnark/std/math/emulated)
• TestSolverConsistency (github.com/consensys/gnark/test)