[NVIDIA] Set proper math type for convolution

[apavliuk55]

Details:

• Sets proper convolution math type according to the math type of the selected algo

[nkogteva]

And what about hardcoded value CUDNN_TENSOR_OP_MATH_ALLOW_CONVERSION, which you try to change previosly? Should we avoid convertion if user specify inference precision == f32?

[apavliuk55]

And what about hardcoded value CUDNN_TENSOR_OP_MATH_ALLOW_CONVERSION, which you try to change previosly? Should we avoid convertion if user specify inference precision == f32?

We tried CUDNN_TENSOR_OP_MATH value but it didn't help to solve the precision issue for some tests with FP32 precision.
So we left the previous value, it won't affect FP16 precision.
For FP32 precision, I think, we should leave it as is because there were no problems with FP32 accuracy with this setting.

[nkogteva]

And what about hardcoded value CUDNN_TENSOR_OP_MATH_ALLOW_CONVERSION, which you try to change previosly? Should we avoid convertion if user specify inference precision == f32?

We tried CUDNN_TENSOR_OP_MATH value but it didn't help to solve the precision issue for some tests with FP32 precision. So we left the previous value, it won't affect FP16 precision. For FP32 precision, I think, we should leave it as is because there were no problems with FP32 accuracy with this setting.

It's not just about tests. If the user has explicitly specified an inference precision f32, we should avoid converting to lower precision, is not it?

[apavliuk55]

And what about hardcoded value CUDNN_TENSOR_OP_MATH_ALLOW_CONVERSION, which you try to change previosly? Should we avoid convertion if user specify inference precision == f32?

We tried CUDNN_TENSOR_OP_MATH value but it didn't help to solve the precision issue for some tests with FP32 precision. So we left the previous value, it won't affect FP16 precision. For FP32 precision, I think, we should leave it as is because there were no problems with FP32 accuracy with this setting.

It's not just about tests. If the user has explicitly specified an inference precision f32, we should avoid converting to lower precision, is not it?

Changed to CUDNN_TENSOR_OP_MATH