Bindings for the new extension system

[vnen]

This is still not 100% functional and needs lots of testing.

This recreates the whole repository to work with the extension system that replaces GDNative in Godot 4.0.

The API is changed to be close to the one used in Godot itself to diminish the differences between an extension and a module (technically you could use the same code for both using a few #ifdefs).

Missing things

• CI checks, including a proper test script. The workflow file is still the same but it isn't backed by the repo scripts.
• Moving math types to inlined structures. For now it uses generated wrappers, without direct member access.
  • Note that the new API JSON file includes sizes and offsets for the builtin types so there should be validation when the inlined structs are done.
• The ClassDB singleton from Godot conflicts with the one from this repository, so for now the engine one isn't available (being skipped during generation). They should be merged somehow (I do have some ideas how to do it but nothing simple so haven't done it yet).
• Linking to godot-headers as a submodule. That repo still needs an update for master so for now this includes the needed files in godot-headers-temp.
• Documentation.
• Probably many more things that I'm forgetting now.

How it works

The README file has a laconic description of the process. Check the demo project in the test folder to have an idea.

You can regenerate the API JSON from Godot by running godot --dump-extension-api. The file extension_api.json will be created in the same folder from which you run the command.

[BastiaanOlij]

@vnen by this time I think it's a good idea if you squash all the commits into one.

For everyone else, while vnen left this as a draft I actually believe we should move to merging this into master sooner rather than later. As the current Godot master no longer works with the old gdnative approach we need to start focusing development on this. Merging this will make it easier for others to submit PRs and help out completing this. It will also open the door to raising issues to discuss further enhancements/short comings and get more people to help out by seeing what needs to be done.

We'll need to create a 3.x branch from the current master and then merging this into master. Everyone can then review their own PRs and decide to either link it to the 3.x branch or to modify it for Godot-cpp 4. That also means we're in line again with Godot development branches.

I don't want to pollute this post any further, we should move this discussion into an issue and link it here. I'll set that up in a minute.
We've had a prelude to this discussion on #scripting on the Godot chat server but I think we only have a subset of our community there so it is worth having the discussion here.

[bruvzg]

Some issues encountered during godotengine/godot#52192 development:

• Casting references fails:

	Ref<TextServerTest> ts;
	ts.instantiate();
	/* void add_interface(const Ref<TextServer> &p_interface); */
	TextServerManager::get_singleton()->add_interface(ts);

src/register_types.cpp:19:54: error: no viable conversion from 'Ref<godot::TextServerTest>' to 'const Ref<godot::TextServer>'
                        TextServerManager::get_singleton()->add_interface(ts);
                                                                          ^~
godot-cpp/include/godot_cpp/classes/ref.hpp:167:2: note: candidate constructor not viable: no known conversion from 'Ref<godot::TextServerTest>' to 'const godot::Ref<godot::TextServer> &' for 1st argument
        Ref(const Ref &p_from) {
        ^
godot-cpp/include/godot_cpp/classes/ref.hpp:171:2: note: candidate constructor not viable: no known conversion from 'Ref<godot::TextServerTest>' to 'godot::TextServer *' for 1st argument
        Ref(T *p_reference) {
        ^
godot-cpp/include/godot_cpp/classes/ref.hpp:177:2: note: candidate constructor not viable: no known conversion from 'Ref<godot::TextServerTest>' to 'const godot::Variant &' for 1st argument
        Ref(const Variant &p_variant) {
        ^
godot-cpp/include/godot_cpp/classes/ref.hpp:113:2: note: candidate function
        operator Variant() const {

• Virtual methods with enum arguments fails:

class TextServerTest : public TextServerExtension {
	GDCLASS(TextServerTest, TextServerExtension);
....
	virtual bool _has_feature(TextServer::Feature feature) override;

In file included from godot-cpp/include/godot_cpp/core/class_db.hpp:38:
In file included from godot-cpp/include/godot_cpp/core/method_bind.hpp:34:
godot-cpp/include/godot_cpp/core/binder_common.hpp:162:59: error: no member named 'convert' in 'godot::PtrToArg<godot::TextServer::Feature>'
        PtrToArg<R>::encode((p_instance->*p_method)(PtrToArg<P>::convert(p_args[Is])...), r_ret);

• Virtual methods with GDNativePtr<> arguments fails:

class TextServerTest : public TextServerExtension {
	GDCLASS(TextServerTest, TextServerExtension);
....
	virtual void _shaped_text_get_glyphs(const RID &shaped, Glyph *gl_array) const override;

In file included from godot-cpp/include/godot_cpp/core/class_db.hpp:38:
In file included from godot-cpp/include/godot_cpp/core/method_bind.hpp:34:
In file included from godot-cppinclude/godot_cpp/core/binder_common.hpp:36:
godot-cpp/include/godot_cpp/core/method_ptrcall.hpp:168:122: error: no member named '___binding_callbacks' in 'godot::Glyph'
                return reinterpret_cast<T *>(godot::internal::interface->object_get_instance_binding(p_ptr, godot::internal::token, T::___binding_callbacks));

[Shatur]

I faced the same problems as @bruvzg. But I also want to add that virtual functions overloads such as _ready() are not called at all.

[bruvzg]

And another inconvenience with the current implementation, there's no clear way to add a custom callbacks for init levels. GDExtensionBinding should have it's (de)initialize methods virtual. And we probably should have extra init levels after singleton registration, or change the init order on the engine side.

[Shatur]

But I also want to add that virtual functions overloads such as _ready() are not called at all.

Double checked, the issue related to the _ready(). Other virtual functions, like _has_point works as expected.

[BastiaanOlij]

@bruvzg so far for me just having Ref<Class> fails, haven't gotten much further with that one though, interesting that you're able to define it but are running into trouble using it.

[bruvzg]

interesting that you're able to define it but are running into trouble using it.

It's building, but instantiate call crashes, memnew also crashes.

[akien-mga]

Commit history somewhat messy and there's too many changes for me to do an in-depth code review, but it's been tested and co-developed by many contributors so it should be good to merge now and refine in master.