nvFuser Python Benchmarks

The python benchmarks use pytest-benchmark and torch.profiler

Usage

To benchmark any target function, use run_benchmark (implemented in core.py):

run_benchmark(benchmark, target_function, function_inputs, rounds=10, warmup_rounds=1)

By default, the target function is run for 10 iterations and 1 warmup iteration.

Running benchmarks

To run a suite of benchmarks, run the benchmark files through pytest. pytest [options] <benchmark-file>.

Some useful options for running benchmarks:

Pytest/Pytest-benchmark options:

• Filtering benchmarks: -k <filter>
• Saving benchmarks: --benchmark-save=NAME, --benchmark-autosave, --benchmark-json=PATH
• Debugging: --benchmark-verbose.

Custom command-line options:

• Disable output validation: --disable-validation Skips the output validation in the nvFuser benchmarks.
• Disable benchmarking: --disable-benchmarking Skips the nvFuser benchmarking, useful for only testing correctness of fusion definitions without benchmarking the fusions.
• Run eager mode benchmarks: --benchmark-eager
• Run torch.compile mode benchmarks: --benchmark-torchcompile