Significantly speed up RNNT loss on CUDA

[titu1994]

Signed-off-by: smajumdar titu1994@gmail.com

What does this PR do ?

Speed up CUDA RNNT loss

Collection: [ASR]

Changelog

• Fix issue with data movement of costs tensor from llForward to pytorch data view in numba
• This alone costs a linear loop (scaling with batch size) that is roughly 10x the kernel costs themselves.
• Fix by writing a small kernel to copy the data and update the costs.

Before your PR is "Ready for review"

Pre checks:

• Make sure you read and followed Contributor guidelines
• Did you write any new necessary tests?
• Did you add or update any necessary documentation?
• Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  • Reviewer: Does the PR have correct import guards for all optional libraries?

PR Type:

• New Feature
• Bugfix
• Documentation

[lgtm-com]

This pull request introduces 1 alert when merging 65be55d into 058fa38 - view on LGTM.com

new alerts:

• 1 for Unused local variable

[lgtm-com]

This pull request introduces 1 alert when merging 961362d into c645c4c - view on LGTM.com

new alerts:

• 1 for Unused local variable

[VahidooX]

LGTM!