Add Benchmarker class and demo notebook for inference throughput benchmarking of Geti models

[ljcornel]

The Benchmarker class is a tool for setting up and managing benchmarks of Geti models on local hardware. At the moment, only benchmarking of the inference rate for the local deployments is supported, but accuracy benchmarking may be added in the future. Currently, the Benchmarker allows the user to do the following:

• Specify a project for which models should be benchmarked (can be either single task or task-chain)
• Specify which model architectures should be included
• Specify which precision levels should be used (i.e. FP16, INT8, ...)
• Specify the media on which the benchmark should run (images from Geti or local images, or a video)

Once this has been specified, the Benchmarker will prepare the project. This means it will check whether all required models and optimized models are available, and request training or optimization of the missing models if not. It will also prepare the required deployments to be able to run the inference locally.

Once prepared, the benchmark can run on any target device (i.e. "CPU" or "GPU"), average inference rates for all deployments will be measured and a results file will be saved to disk.

The notebook 011_benchmarking_models is a tutorial that shows how to initialize, prepare and run the benchmark.