Implement a way to use deep neural networks in Godot

[aabmets]

PyTorch, SciKit-Learn, TensorFlow, etc. can be used to create responsive and evolving AI actors
into computer games to provide greater interactivity and deeper gameplay for players.

Godot has the opportunity to become on-par with Unity Engine, where they are already being widely used.

Therefore I propose to integrate a new feature into Godot: the ability to use deep neural network frameworks like TensorFlow to build, train and test DNN models and use them for making in-game predictions.

I've proposed this as an official addon, because most machine learning frameworks use Python, C++, JavaScript, or Java to operate and GDNative is ridiculously difficult for your average developer to use due to lack of documentation. Also, unofficial Python bindings for Godot require the user to compile the Godot Engine, which is just not something anybody can do to create a reliable and working binary.

[Anutrix]

An official module/addon/plugin would be good.

Note:

"I mentioned that GDNative is not a language. It loads shared libraries. These shared libraries can be written in any language you want. Godot just needs to know how to use them. Because of that, there's no code editor for GDNative in the Godot editor. Because there is no source code. There only is a shared library, nothing more, nothing less. So you have to write your code outside of the editor."

Source:https://godotengine.org/article/dlscript-here (Really really old article)

Documentation might be lacking(not sure) but it just needs time.
Some info for now: https://github.com/GodotNativeTools/godot_headers/blob/master/README.md, https://stackedboxes.org/2017/08/20/trying-out-gdnative/

Also, similar attempts on this have been done before:
https://github.com/Dariasteam/Geode
https://goblinrefuge.com/mediagoblin/u/darias/m/evolving-neural-networks-for-racing-agents/
smart-design.kiev.ua/7lus/python-ascii-game-engine.html
(dead link but google search had this snippet "An engine for small games for reinforcement learning agents. .... This essay introduces a deep learning framework for creating autonomous agents .... a Godot contributor and Python lover who develops a GDNative interface to use Python 3 as ...")

[fire]

https://github.com/godot-extended-libraries/godot-tensorflow-workspace For reference.

[aabmets]

Great, I wasn't aware of that project.
Still, though, I noticed that it requires compiling Godot and it uses Python 2, which is becoming unsupported.

[fire]

The resulting project doesn't use the python runtime.

[hackini]

I think it's important to clarify things a bit, because for instance "Integrating Tensorflow in Godot for training" makes absolutely no sense to me.

1. First, supposing we are not taking about Reinforcement Learning but classic ML
   (so just your Neural Net train & inference)

Then it is separated into two very distinct and independent phases : train and inference.

For training :

• It make no sense to bind Tensorflow, Caffe, PyTorch, to Godot for the following reasons :
  • Training requires you to write code that defines the model (the compute/autograd graph). This is done generally in Python (or Julia, Matlab, R, etc.), this has no dependency or anything to do with Godot, so it makes no sense to write that from Godot.
  • Training requires data, where does the data comes from ? :
    • From outside your game ? (Denoiser, mesh generation, texture generation, audio generation, voice recognition, etc. ) then you don't care about Godot.
    • From inside your game ? Then it's 2 lines of code to just write your data to a .csv, once again we don't care about Godot.

For Inference :

• A surprisingly not so well known fact : PyTorch, Tensorflow, etc. are not interesting for inference in production, they are used for training. In the industry people do not use those much for inference in production, they rather use :
  • inference engines (like here or there). Those are dedicated to support lots of platform (not CUDA only like tensorflow), have special optimization to use the best of the hardware and what's available (SIMD, Vulkan, GLSL, CUDA, special instructions, NEON, OpenCL, etc.).
  • Or, in the future, we can expect the OS / Platform to offer Native ML APIs i.e an integrated inference engines in the OS (Android & IOS are already offering / working on that if I'm not mistaken, and there is a proposal of an API for the Web too)

Conclusion :

All we want is just bind an inference engine to Godot (ideally abstract the API enough so we can have different inference engine / Native ML APIs for different platforms), but we don't care about Tensorflow, Caffe, PyTorch, PlaidML, SkLearn, etc. Moreover, different people work with different framework anyways, so not a good idea to impose any framework especially when we don't need to.

To give an analogy : Godot does not support "Gimp" or "Photoshop", it supports ".png". Same for ML : Godot should support "ONNX" and/or "NNEF" not "Tensorflow" or "PyTorch".

2. Supposing we are taking about Reinforcement Learning

Now here the training and inference phase are less independent.

Using on-policy & episodic (for eg REINFORCE, Monte-Carlo, episodic Actor-Critic, etc.) you typically run one episode and collect data, then update your policy/value network and repeat. Other algo may update the policy at each step instead.

So now we need to alternate between Godot (perform one episode and collect data) and PyTorch, Tensorflow, etc. To do so, a very badly documented standard exists : Gym by OpenAI.

What we can do is wrap Godot as an Gym environment (over IPC). I have been prototyping such a feature and I have things starting to work. However I still need an inference engine for using it in production.

Sry for the long post but in TL:DR :

• Tensorflow, PyTorch, Sklearn, PlaidML, etc. are good for training. They are less interesting for in-production inference, and that's okay because they are made for research, testing and developing models.
• Trained models can be exported to ONNX or NNEF which can then be executed efficiently, in real-time and across all platform either using dedicated inference engines (onnxruntime, MACE, etc.) or the OS Native ML APIs.
• Training will not happen in-game (you are not going to do a full backprop of millions of neurons in real-time in parallel of your game running) so it doesn't make sense to try to integrate training frameworks to Godot.
• Data is either external to the game or can be exported from the game. The only special case is Reinforcement Learning where train and inference are interlaced. But then we should look at Gym + IPC (or maybe something to do with shared memory + GDNative to avoid communication overhead). I've been playing around with this idea and got encouraging results but I cannot use it in production because I still need an inference engine to execute the learned model in game without IPC & Python overhead.

So basically, in my opinion all we need is just an abstract enough API over a GDNative bind of an inference engine. The API would be basically load and then execute over input tensor :

model = loadTrainedModel(myONNXModel)
ouput = model.predict(input)

[fire]

You seem to be requesting that ONNX be the model and that https://github.com/microsoft/onnxruntime or https://github.com/XiaoMi/mace be the exact engine.

The tensorflow-lite seems to slot in the same place compared to tensorflow (full).

[fire]

The ideal engine is one that is small enough to be embedded into the engine or a gdnative plugin and can function with opencl, cuda, cpu, accelerated apis or compute shaders.

It has to be popular enough that any random model can be converted to it.

It should be able to be compiled for all the platforms godot supports. At least desktop and mobile.

[hackini]

You seem to be requesting that ONNX be the model and that https://github.com/microsoft/onnxruntime or https://github.com/XiaoMi/mace be the exact engine.

The tensorflow-lite seems to slot in the same place compared to tensorflow (full).

More like : ONNX the model format and then a Godot API that abstracts over any inference engine / Native ML API that we choose to use underneath.

MACE / onnxruntime are just examples, it could be also Tensorflow-lite or ncnn or any other that fit the best the requirements you've mentioned.

ncnn seems to fit those requirements rather well actually (small, embeddable, support ONNX and thus any popular framework can export to it, pure c++, not CUDA-only but uses also Vulkan or CPU special instructions when available, etc...).

I think the Godot API should be simple and define only one supported format rather than arbitrarily implement dozens of formats to support each framework. This way we can even use different backend engines if needed without breaking the user's code and support most ML training frameworks.

ONNX seems clearly the winner format here as all main frameworks can export to it (I would personally prefer NNEF as it's backed by Khronos rather than Facebook, Tencent, MS, etc. but nobody seem to support it, so ONNX seems the best choice)

[hackini]

Basically we should define :

• which format the Godot API accepts (so here ONNX seems the most widely supported by all framework but maybe I missed some other widely supported formats ?)
• which engine to use : one small engine for all platform ? different ones for each platform ? which ones ? Native OS APIs ? I don't know what's the best choice but as long as the API abstracts over those, we can always experiment with one engine and later change or use other engine for other platform without breaking the user code.

[fire]

If someone wants it right now they can convert nccn to a Godot module, if not maybe over the week I could look into it.

[fire]

Nccn's issue tracker seems to be full of unresolved issues, is there a next best alternative?

[aabmets]

I'm especially interested in using reinforcement learning to make the game adapt to players choices in-game.
Example: to make NPC enemies more dangerous or create a storyteller like in RimWorld, but much more powerful.
For RL, you need to be able to train the model while the game is running, therefore I propse that no limitations be placed on how it's possible to use a NN framework in Godot, but make it the obligation of developer to use it in a wise manner which does not break their game.

[aabmets]

When it comes to the data formats and API-s, I think the most compatible solution would be to use HDF5 file format for storing models and/or weights and the Keras API for talking with backend NN frameworks, which support the Keras specification.

[groud]

Most games do not require any kind of deep neural networks. It might be a cool feature, but without a common use case it's very unlikely it will be added to the core engine.

Even for AI in general, it is unlikely we ever add any kind of AI framework (state machines, behavior trees...). There are too many ways to build such AI system, so it would not make much sense to force a solution onto another.

[kruglinski]

Most games do not require any kind of deep neural networks. It might be a cool feature, but without a common use case it's very unlikely it will be added to the core engine.

Even for AI in general, it is unlikely we ever add any kind of AI framework (state machines, behavior trees...). There are too many ways to build such AI system, so it would not make much sense to force a solution onto another.

I agree, Game AI is all about fun for players, not for super realistic or high precision predictions, therefore Game "AI" is not the "AI" from people talks today. if your AI make player like a fool and beat the shit out of player, your game will fail!

I uses many "AI" technique from the book "Programming Game AI by Example", combine some simple AI we can get a very complex game AI.

I think the best way to implement these AIs is to implement them in high level language like py, gdscript, focus on algorithm/game logic not the specific, especially don't integrated them in engine core

If Godot must provider game AI framework, I hope it is collection of gdscripts

[ElGatoNinja]

Most games do not require any kind of deep neural networks. It might be a cool feature, but without a common use case it's very unlikely it will be added to the core engine.
Even for AI in general, it is unlikely we ever add any kind of AI framework (state machines, behavior trees...). There are too many ways to build such AI system, so it would not make much sense to force a solution onto another.

I agree, Game AI is all about fun for players, not for super realistic or high precision predictions, therefore Game "AI" is not the "AI" from people talks today. if your AI make player like a fool and beat the shit out of player, your game will fail!

I uses many "AI" technique from the book "Programming Game AI by Example", combine some simple AI we can get a very complex game AI.

I think the best way to implement these AIs is to implement them in high level language like py, gdscript, focus on algorithm/game logic not the specific, especially don't integrated them in engine core

If Godot must provider game AI framework, I hope it is collection of gdscripts

I don't think that the point of Ai is programming intelligent game enemies. But for example predicting a realistic blend between 3D animations or adapting music to what is happening in the screen.

[fire]

I don't have much bandwidth for this, but it should be possible to do a non-core module for Godot that uses the NCNN ai library and ONNX models via waifu2x to scale images.

https://github.com/nihui/waifu2x-ncnn-vulkan

Here is an example scaling the input and output twice as big:

The plan is to integrate NCNN and think of an interface for the onnx models. It doesn't need to be much, just the ability to load the ONNXmodels via a resource. I've already done this for Tensorflow Lite. https://github.com/godot-extended-libraries/godot-tensorflow/blob/master/loader_tflite.h#L39

As stated above the NCNN is useful because it has GPU acceleration WITHOUT installing NVIDIA CUDA and supports multiple platforms.

[GeorgeS2019]

@hackini @aspenforest SciSharp has many options to apply deep learning in ways similar to python.
One particular use case is reinforced learning of 3D skeletal/vertex agents (from Godot) using OpenAI/GYM but C# .NET

[GeorgeS2019]

Instead of using ONNX with ML.NET, it seems straight forwards just to use SciSharp TensorFlow.NET or Touch.NET

[GeorgeS2019]

Please see the option for .NET Deep Learning for Godot use cases

[gururise]

There's also GodotGymAI that uses PyTorch, which allows for both TRAINING and INFERENCE within godot.

As mentioned above, training inside Godot would be very useful, as it would allow enemy AI to train/learn within the confines of a game environment. This is similar to what Unity is doing with it's ML Agents project.

[Xrayez]

Please forgive my ignorance if what I'm saying does not really apply to the topic.

I guess it's safe to say that we won't see any kind of neural network implementation in Godot Engine being integrated in core any time soon. If it does, I think this should take around 10-20 years for the use cases to become common enough among developers, but at that time we'll likely be using some other neural-network-based game engine. 🙂

I do believe that a general-purpose game engine should make the development of not-so-common features easier on the module/plugin avenue at the very least.

I'm only scratching the surface around the topic myself, I'm especially interested in reinforcement learning field though. I might be wrong but it seems to me that RL by definition does not/should not rely on anything which hardcodes AI behavior. The ideal RL system should only care about the environment, agents, and the policy. So, the behavior is largely determined by environment and some kind of an input. As humans we have to use a medium such as a joypad. Likewise, the RL agents should also interact via a very similar medium (represented virtually as input states).

I've been thinking that it might make sense for the engine to provide such medium seamlessly to RL agents. This lengthy introduction leads to something which I've been working on, the use cases not necessarily motivated by RL but still I think this might be useful for RL and/or input-driven AI, if interested read my proposal at godotengine/godot-proposals#104.

Talking specifically, I think exposing the input state as some kind of a decoupled data structure should help the learning process of RL agents, especially if we talk about multi-agent RL because we need to have independent, local input state instances for each agent which wouldn't interfere with the global state, especially important if we want to match the exact input medium which humans use to control the agents themselves.

If the proposed feature is at least marginally useful for this, that's a win in my book. 👀

[strank]

For the reinforcement learning use case, the GodotGymAI looks like a great start, but it might also be worthwhile looking into the communication setup that Unity's ml-agents package uses. If we could copy that (i.e. implement the RPC communicator / Agent classes / ... for godot) and implement a GodotEnvironment version of their UnityEnvironment python class, the whole training setup of that package could be used.

[mhilbrunner]

This sounds great, but very much like plugin territory to me.

[bhack]

I think It Is interesting also the inverse.
So to use Godot to generate synth datasets like https://unrealcv.org/ and others projects.

[GeorgeS2019]

I am c# developer, I plan to do deep learning through the Godot c# deep learning way,

@bhack It is also possible to use OpenCVSharp in Godot

Any c# developer here? us Unity AI agent concepts and port that to godot

[mhilbrunner]

Closing, as this seems more like plugin/module territory. Thanks for the interesting discussion :)

[Xrayez]

Somewhat similar proposal opened in godotengine/godot-proposals#1577.

[emmaccode]

I don't have much bandwidth for this, but it should be possible to do a non-core module for Godot that uses the NCNN ai library and ONNX models via waifu2x to scale images.

https://github.com/nihui/waifu2x-ncnn-vulkan

Here is an example scaling the input and output twice as big:

The plan is to integrate NCNN and think of an interface for the onnx models. It doesn't need to be much, just the ability to load the ONNXmodels via a resource. I've already done this for Tensorflow Lite. https://github.com/godot-extended-libraries/godot-tensorflow/blob/master/loader_tflite.h#L39

As stated above the NCNN is useful because it has GPU acceleration WITHOUT installing NVIDIA CUDA and supports multiple platforms.

I think you make a valid point... However, it would be cool to have serialized NNs that represent different stages of difficulty with more or less dimensional data...
I am very interested, not necessarily for the ML aspect, but moreso because I love Julia so much I'd love to see a Godot implementation of the language... I might start working on this, but alas! I am knee deep in preparing for a breaking release of my own ML package -- so we will have to see when I have the time! I know this issue is closed, but I think that Julia's parallelism, speed, JIT, and of course the fact that it uses LLVM could make it a great choice for implementing some crazy complex algorithms... I mean the language is declarative and dynamic like GDScript, but runs at the speed of C. While GDScript can handle most programming with C below it I think, it obviously can't fit the bill for a lot of the crazier things one might want to do... Furthermore, it would be a cool ecosystem to see underneath Godot.

[fire]

Community update, I have archived my tensorflow repo.

https://lupoglaz.github.io/GodotAIGym/tutorial_deploy.html is the most promising ai integration. It's Linux only, so you will need to convert the AI from shared memory to something like zeromq.