[Performance] Faster RNNs #8470

Job	Run time
build-wheel-linux (3.8, cp38-cp38, --index-url https://download.pytorch.org/whl/cpu, "['cpu', '11...	23s
build-wheel-mac (3.8, 3.8)	35s
build-wheel-linux (3.9, cp39-cp39, --index-url https://download.pytorch.org/whl/cpu, "['cpu', '11...	25s
build-wheel-mac (3.9, 3.9)	30s
build-wheel-windows (3.8, 3.8)	28s
build-wheel-linux (3.10, cp310-cp310, --index-url https://download.pytorch.org/whl/cpu, "['cpu', ...	25s
build-wheel-mac (3.10, 3.10.3)	21s
build-wheel-windows (3.9, 3.9)	29s
build-wheel-linux (3.11, cp311-cp311, --index-url https://download.pytorch.org/whl/cpu, "['cpu', ...	24s
build-wheel-mac (3.11, 3.11)	38s
build-wheel-windows (3.10, 3.10.3)	19s
build-wheel-windows (3.11, 3.11)	31s
test-wheel-windows	1s
test-wheel	1s
	5m 30s

Provide feedback