Benchmark for image-based neural networks for inference framework

[sidgoyal78]

TODO:

The main task is to benchmark inference API for image recognition models, using different batch sizes and run on either GPU and CPU. We want to do both a overall time and operator level timing benchmark.

• Model in recognize digits
  Results are here:https://github.com/sidgoyal78/paddle_notes/blob/master/benchmark/recoginze_digits.md
  Code used for those results: PR [Do not merge] Profiling inference unit-test for recognize digits #8497

• VGG-16

• Resnet

• Googlenet

[Xreki]

We need to compare the performance with TensorRT. Do you test the inference time of TensorRT for the same model?

[sidgoyal78]

It is still work in progress. I got a machine setup for running TensorRT today. I will have some results in the next 2 days.