GPU acceleration

[kenthinson]

I see in the docs that GPU acceleration is only available for CUDA on Linux. Is there any plans to support Apples Metal API so we can get GPU acceleration on AMD / Intel GPU'S? or maybe something like nGraph+ PlaidML?

[BradLarson]

We currently layer on top of the eager-mode TensorFlow runtime, so we support the accelerators that the TensorFlow runtime does. While there are community-supported TensorFlow builds for AMD ROCm, that unfortunately only provides acceleration on Linux.

As has been discussed during the design meetings and in the mailing list, we are currently working to layer the higher-level APIs in such a way that we can decouple from the current eager-mode TensorFlow runtime and explore alternative underlying implementations.

These alternatives may provide opportunities to expand accelerator compatibility. For example, Ed Connell has presented during the community meetings about his SwiftRT project which has support for Vulkan. However, these experiments are in the early stages now and don't currently provide a replacement for the eager-mode TensorFlow runtime.

[kenthinson]

@BradLarson thanks for the information.

[maxscience]

Hi any updates on this? I've watched the last Swift4TF video call but there was no mention of Metal support and GPU acceleration on modern Macs.

[BradLarson]

@maxscience - Expanding beyond the eager-mode TensorFlow runtime gives us additional options for how we can access accelerators. For example, the tracing architecture used in x10 could at some point possibly be used with a Metal backend, if the effort was invested to build one out. We're also working on other runtime implementations that could potentially allow for this.

It has taken a lot of work to get to the stage where we can start decoupling from the TensorFlow eager runtime, so just getting to this point has been our focus. In the near term, x10 will provide compatibility with TPUs and should significantly improve performance on Linux-hosted Nvidia GPUs. It will be interesting to see where we can go beyond that.